Methods and systems for displaying additional content on a heads up display displaying a virtual reality environment

ABSTRACT

Methods and systems for enabling a user to perform a full body movement while viewing a virtual reality environment on a head up displays without interfering with viewing of content on the heads up display. Specifically, a full body movement of the user is detected. In response to detecting the full body movement, the additional content is generated for display in a portion of the virtual reality environment corresponding to a foreground area of the user&#39;s visual field. The additional content assists the user perform the full body movement.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/140,272, filed Apr. 27, 2016 (allowed). The disclosure of which ishereby incorporated by reference herein in its entirety.

BACKGROUND

In conventional systems, it may desirable to display additional content(e.g., stock price information, sports score information, newsinformation, weather information, a clock, a schedule of events) inaddition to main content (e.g., a movie, a television show, a videogame, a virtual reality world, a media guidance screen) on a display.Unfortunately, displaying additional content runs the risk that theadditional content obscures or otherwise interferes with the display ofthe main content. This problem may be especially relevant in virtualreality environments, where systems put an emphasis on an immersiveexperience.

SUMMARY

Accordingly, methods and systems are described herein that re-positionadditional information (e.g., stock price information, sports scoreinformation, news information, weather information, a clock, a scheduleof events) around the main content (e.g., a movie, a television show, avideo game, a virtual reality world, a media guidance screen) based onthe needs of a user in a virtual reality environment. For example,control circuitry as described herein may limit the additionalinformation to a peripheral area of the visual field of the user, andmay only re-position the additional information in response todetermining that the user wishes to leave his or her immersion in thevirtual reality environment. Specifically, the control circuitrydetermines a visual field of the user based on the movement of thecenter of gaze. If the control circuitry determines that the portion ofthe virtual reality environment in which the additional content isgenerated for display corresponds to a foreground area of the user'svisual field, the control circuitry generates for display the additionalcontent in a portion of the virtual reality environment corresponding toa peripheral area of the user's visual field. By ensuring that thecontrol circuitry generates for display the additional content in aportion of the virtual reality environment that corresponds to aperipheral area of the visual field of the user, it is possible toensure that the additional content does not interfere with the user'sviewing of main content corresponding to foreground areas of the visualfield of the user.

Specifically, in response to detecting a full body movement, the controlcircuitry generates for display the additional content in a portion ofthe virtual reality environment corresponding to a foreground area ofthe user's visual field. The additional content assists the user toperform the full body movement. By generating for display the additionalcontent in a portion of the virtual reality environment that correspondsto a foreground area of the user's visual field, it is possible toensure that the user can easily see the additional content. Theadditional content is configured to assist the user in performing thefull body movement without interfering with the user's viewing of maincontent on the heads up display. For example, if the user is walking,the additional content may be a video of the physical surroundings ofthe user that can assist the user in, for example, avoiding obstacles inthe physical surroundings without needing to stop viewing content on theheads up display. Thus, generating for display the additional content inthe foreground area enables the user to perform the full body movementwithout interfering with the user's viewing of content on the heads updisplay.

In some aspects, methods and systems are provided herein for presentingadditional content in virtual reality environments on heads up displaysshowing main content without interfering with a user's viewing of themain content.

The main content may be any content that is intended to be the object ofa user's main focus. For example, the main content may be a media asset,such as a movie, a television show, a video game, or a virtual realityworld. As another example, the main content may be a media guidancescreen.

The additional content may be any content that is not main content. Theadditional content may be unrelated to the main content or related tothe main content. For example, the additional content may be a video ofthe user's physical surroundings, stock price information, sports scoreinformation, news information, weather information, a clock, or aschedule of events.

The virtual reality environment may be any non-physical contentdisplayed to a user in such a way that the non-physical content appearsto the user to have a semblance of physicality. For example, the virtualreality environment may be a virtual world (for example, a virtual worldin a game) which appears to the user to be the world in which the useris located. As another example, the virtual reality environment may benon-physical content that appears to the user to be superimposed on thephysical world. For example, the virtual reality environment may be aspeedometer display (or any other display) that is superimposed on whatthe user sees through the windshield of his or her car (or any othertransparent surface). As another example, the virtual realityenvironment may be a media asset (for example, a television show or amovie) presented to the user such that the display of the media assetfully encompasses the visual field of the user.

The heads up display may be any display capable of displayingnon-physical content to a user in such a way that the non-physicalcontent appears to the user to have a semblance of physicality. Forexample, the heads up display may be a head-mounted display that fullycovers the eyes of the user. The head-mounted display may be configuredas eyeglasses, binoculars, a helmet, etc. As another example, the headsup display may be a display (for example, a display integrated with awindshield or eyeglasses) that superimposes non-physical content on aview of the physical world which the user can see through the heads updisplay. As another example, the heads up display may be a room in whichthe user is located, where the walls of the room are covered in displayscreens.

The methods and systems include control circuitry configured to generatefor display, in a first portion of a virtual reality environment in aheads up display, first main content, wherein the first portioncorresponds to a foreground area of a first visual field of a user.

The control circuitry may be based on any suitable processing circuitry,such as circuitry based on one or more microprocessors,microcontrollers, digital signal processors, programmable logic devices,field-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), etc., and may include a multi-core processor (e.g.,dual-core, quad-core, hexa-core, or any suitable number of cores) orsupercomputer. The processing circuitry may be distributed acrossmultiple separate processors or processing units, for example, multipleof the same type of processing units (e.g., two Intel Core i7processors) or multiple different processors (e.g., an Intel Core i5processor and an Intel Core i7 processor).

The first main content may be any portion of main content. The firstmain content may be a subset of the main content. For example, the firstmain content may be one or more objects or characters in a media asset.As another example, the first main content may be a particular areawithin a virtual world. As another example, the first main content maybe a specific portion of a media guidance screen.

The first portion of the virtual reality environment may be any portionof the virtual reality environment. The first portion may be a subset ofthe virtual reality environment. The first portion may be a top, bottom,right, or left portion of the virtual reality environment. The firstportion may be approximately a majority or approximately a minority ofthe virtual reality environment.

A visual field of the user may be anything that a user can see when theuser is in a specific position. For example, the visual field may bedetermined based on movements of the user's head. As another example,the visual field may be determined based on movements of the user'scenter of gaze. For example, the visual field of the user may encompassareas within a first number of degrees to the right and left of theuser's center of gaze, a second number of degrees above the center ofgaze, and a third number of degrees below the center of gaze. Forexample, the first number of degrees may be equal to or greater than 95degrees, for example, 95 degrees, 100 degrees, 105 degrees, 110 degrees,115 degrees, 120 degrees, >120 degrees, or any suitable number ofdegrees. Alternatively, the first number of degrees may be less than 95degrees, for example, 90 degrees, 85 degrees, 80 degrees, 75 degrees, 70degrees, <70 degrees, or any suitable number of degrees. For example,the second number of degrees may be equal to or greater than 60 degrees,for example, 65 degrees, 70 degrees, 75 degrees, 80 degrees, 85degrees, >85 degrees, or any suitable number of degrees. Alternatively,the second number of degrees may be less than 60 degrees, for example,55 degrees, 50 degrees, 45 degrees, 40 degrees, <40 degrees, or anysuitable number of degrees. For example, the third number of degrees maybe equal to or greater than 70 degrees, for example, 75 degrees, 80degrees, 85 degrees, 90 degrees, >90 degrees, or any suitable number ofdegrees. Alternatively, the third number of degrees may be less than 70degrees, for example, 65 degrees, 60 degrees, 55 degrees, <55 degrees,or any suitable number of degrees. The visual field may be the portionsof a screen of a head-mounted display that the user can see. The visualfield may vary from user to user and may depend on visual impairmentsspecific to each user.

A foreground area of a user's visual field may be any portion of theuser's visual field that the user can see with normal vision. Theforeground area may encompass a subset of the visual field. Theforeground area may encompass areas of the visual field that aresubstantially in the center of the user's visual field. The foregroundarea may be determined based on movements of the user's head. As anotherexample, the foreground area may be determined based on movements of theuser's center of gaze. For example, the foreground area may be within afourth number of degrees to the right and left of the center of gaze ofthe user, a fifth number of degrees above the center of gaze of theuser, and a sixth number of degrees below the center of gaze of theuser. For example, the fourth, fifth, and sixth numbers of degrees maybe equal to or greater than 18 degrees, for example, 20 degrees, 25degrees, 30 degrees, 35 degrees, >35 degrees, or any suitable number ofdegrees. Alternatively, the fourth, fifth, and sixth numbers of degreesmay be less than 18 degrees, for example, 15 degrees, 10 degrees, 5degrees, <5 degrees, or any suitable number of degrees. The fourth,fifth, and sixth numbers of degrees may be different. The foregroundarea may be the portions of a screen of a head-mounted display that theuser can see with main vision. The foreground area may vary from user touser and may depend on visual impairments specific to each user.

The control circuitry is further configured to generate for display, ina second portion of the virtual reality environment in the heads updisplay, second main content and additional content, wherein the secondportion corresponds to a peripheral area of the first visual field ofthe user.

The second portion of the virtual reality environment may be any portionof the virtual reality environment. The second portion may be a subsetof the virtual reality environment. The second portion may be differentthan the first portion. The second portion may be a top, bottom, right,or left portion of the virtual reality environment. The second portionmay be near or at the boundary or corner of the virtual realityenvironment. The second portion may be approximately a majority orapproximately a minority of the virtual reality environment.

The second main content may be any portion of main content. The secondmain content may be a subset of the main content. For example, thesecond main content may be one or more objects or characters in a mediaasset. As another example, the second main content may be a particulararea within a virtual world. As another example, the second main contentmay be a specific portion of a media guidance screen.

A peripheral area of a user's visual field may be any portion of auser's visual field that the user can see with peripheral vision. Theperipheral area may be areas that are substantially at the edges of theuser's visual field. The peripheral area may be determined based onmovements of the user's head. As another example, the peripheral fieldmay be determined based on movements of the user's center of gaze. Forexample, the peripheral area may be any portion of the visual field thatis not foreground area. For example, the peripheral area may encompassareas of the visual field within a certain number of degrees of theouter boundary of the visual field. The peripheral area may be theportions of a screen of a head-mounted display that the user can seewith peripheral vision. The peripheral area may vary from user to userand may depend on visual impairments specific to each user.

The control circuitry is further configured to determine a second visualfield of the user based on a movement of a center of gaze of the user.For example, the control circuitry may determine the second visual fieldby determining the new center of gaze. For example, the controlcircuitry may determine the second visual field by detecting a movementof the head of the user.

The control circuitry is further configured to determine that the secondportion corresponds to a foreground area of the second visual field. Forexample, the second portion may correspond to a corner of the firstvisual field but correspond to the center of the second visual field.

The control circuitry is further configured, in response to determiningthat the second portion corresponds to the foreground area of the secondvisual field, to generate for display the additional content in a thirdportion of the virtual reality environment, wherein the third portioncorresponds to a peripheral area of the second visual field.

The third portion of the virtual reality environment may be any portionof the virtual reality environment. The third portion may be a subset ofthe virtual reality environment. The third portion may be different thanthe first and second portions. The third portion may be a top, bottom,right, or left portion of the virtual reality environment. The thirdportion may be near or at the boundary or corner of the virtual realityenvironment. The third portion may be approximately a majority orapproximately a minority of the virtual reality environment.

The methods and systems further include a detection module configured todetect the movement of the center of gaze of the user. For example, thedetection module may detect the movement of the center of gaze bydetecting a gaze point of each eye of the user using light. For example,the detection module may detect the movement of the center of gaze bydetecting a movement of the head of the user. For example, the detectionmodule may detect the movement of the center of gaze by detecting achange in a video of the physical surroundings of the user with capturedwith a camera physically coupled to the user. The center of gaze may beany area of the user's visual field to which the gaze of the user issubstantially focused. The center of gaze may be a central portion ofwhat a user can see. The center of gaze may be a midpoint between thegaze point of each eye of the user. In users with a visual impairment inone eye, the center of gaze may be the gaze point of the unimpaired eye.

In some embodiments, the detection module is further configured todetect the movement of the center of gaze by detecting the center ofgaze of the user. In certain embodiments, the detection module isfurther configured, when detecting the center of gaze of the user, totransmit light to each eye of the user; collect an image of each eye ofthe user; detect, in each image, a location of a reflection in an eye ofthe user; determine a location of each pupil of the user; compare thelocation of each pupil to the location of each reflection; based oncomparing the location of each pupil to the location of each reflection,determine a gaze point of each eye of the user; and determine the centerof gaze by determining a midpoint between the gaze point of each eye ofthe user.

In certain embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the head-mounted displayincludes an accelerometer, and the detection module is furtherconfigured, when detecting the movement of the center of gaze of theuser, to detect, by the accelerometer, an acceleration of the head ofthe user.

In some embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the head-mounted displayincludes a camera feeding a video of the user's physical surroundings tothe display, and the control circuitry is further configured, whengenerating for display the additional content, to generate for displaythe video of the user's physical surroundings.

In certain embodiments, the control circuitry is further configured,when generating for display the additional content, to generate fordisplay the additional content as a picture-in-picture.

In some embodiments, the control circuitry is further configured, whendetermining the second visual field of the user based on the movement ofthe center of gaze, to determine a new center of gaze based on themovement of the center of gaze and to determine an area that is within afirst number of degrees to the right and left of the new center of gaze,a second number of degrees above the new center of gaze, and a thirdnumber of degrees below the new center of gaze.

In certain embodiments, the control circuitry is further configured,when determining that the second portion corresponds to the foregroundarea of the second visual field, to determine the foreground area of thesecond visual field, and the control circuitry is further configured,when determining the foreground area of the second visual field, todetermine an area that is within a fourth number of degrees to the rightand left of the new center of gaze, a fifth number of degrees above thenew center of gaze, and a sixth number of degrees below the new centerof gaze, wherein the fourth number is smaller than the first number, thefifth number is smaller than the second number, and the sixth number issmaller than the third number.

In some embodiments, the systems and methods further include a userinterface configured to receive an input from the user, and the controlcircuitry is further configured, in response to the input, to generatefor display the additional content in a portion of the virtual realityenvironment corresponding to a foreground area of the second visualfield.

In certain embodiments, the control circuitry is further configured,when generating for display the additional content, to measure a timeperiod after the movement of the center of gaze of the user during whichthe center of the gaze of the user has not substantially moved;determine that the time period is greater than a threshold time period;and in response to determining that the time period is greater than thethreshold time period, generate for display the additional content.

In some aspects, methods and systems are provided herein for enablingusers to perform full body movements while viewing virtual realityenvironments on heads up displays without interfering with viewing ofcontent on the heads up displays.

The virtual reality environment may be any non-physical contentdisplayed to a user in such a way that the non-physical content appearsto the user to have a semblance of physicality. For example, the virtualreality environment may be a virtual world (for example, a virtual worldin a game) which appears to the user to be the world in which user islocated. As another example, the virtual reality environment may benon-physical content that appears to the user to be superimposed on thephysical world. For example, the virtual reality environment may be aspeedometer display (or any other display) that is superimposed on whatuser sees through the windshield of his or her car (or any othertransparent surface). As another example, the virtual realityenvironment may be a media asset (for example, a television show or amovie) presented to the user such that the display of the media assetfully encompasses the visual field of the user.

The heads up display may be any display capable of displayingnon-physical content to a user in such a way that the non-physicalcontent appears to the user to have a semblance of physicality. Forexample, the heads up display may be a head-mounted display that fullycovers the eyes of the user. The head-mounted display may be configuredas eyeglasses, binoculars, a helmet, etc. As another example, the headsup display may be a display (for example, a display integrated with awindshield or eyeglasses) that superimposes non-physical content on aview of the physical world which the user can see through the heads updisplay. As another example, the heads up display may be a room in whichthe user is located, where the walls of the room are covered in displayscreens.

The full body movement may be any physical movement by a user thatrequires movement of a substantial portion of the user's entire body.For example, the full body movement may be walking, jumping, standingup, sitting down, rotating one's body, etc.

The systems and method include control circuitry configured to generatefor display, in a first portion of a virtual reality environment in aheads up display, main content, wherein the first portion corresponds toa foreground area of a visual field of a user.

The control circuitry may be based on any suitable processing circuitry,such as circuitry based on one or more microprocessors,microcontrollers, digital signal processors, programmable logic devices,field-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), etc., and may include a multi-core processor (e.g.,dual-core, quad-core, hexa-core, or any suitable number of cores) orsupercomputer. The processing circuitry may be distributed acrossmultiple separate processors or processing units, for example, multipleof the same type of processing units (e.g., two Intel Core i7processors) or multiple different processors (e.g., an Intel Core i5processor and an Intel Core i7 processor).

The first portion of the virtual reality environment may be any portionof the virtual reality environment. The first portion may be a subset ofthe virtual reality environment. The first portion may be a top, bottom,right, or left portion of the virtual reality environment. The firstportion may be approximately a majority or approximately a minority ofthe virtual reality environment.

The main content may be content that is intended to be the object of auser's main focus. For example, the main content may be a media asset,such as a movie, a television show, a video game, or a virtual realityworld. As another example, the main content may be a media guidancescreen.

A visual field of the user may be anything that a user can see when theuser is in a specific position. For example, the visual field may bedetermined based on movements of the user's head. As another example,the visual field may be determined based on movements of the user'scenter of gaze. For example, the visual field of the user may encompassareas within a first number of degrees to the right and left of theuser's center of gaze, a second number of degrees above the center ofgaze, and a third number of degrees below the center of gaze. Forexample, the first number of degrees may be equal to or greater than 95degrees, for example, 95 degrees, 100 degrees, 105 degrees, 110 degrees,115 degrees, 120 degrees, >120 degrees, or any suitable number ofdegrees. Alternatively, the first number of degrees may be less than 95degrees, for example, 90 degrees, 85 degrees, 80 degrees, 75 degrees, 70degrees, <70 degrees, or any suitable number of degrees. For example,the second number of degrees may be equal to or greater than 60 degrees,for example, 65 degrees, 70 degrees, 75 degrees, 80 degrees, 85degrees, >85 degrees, or any suitable number of degrees. Alternatively,the second number of degrees may be less than 60 degrees, for example,55 degrees, 50 degrees, 45 degrees, 40 degrees, <40 degrees, or anysuitable number of degrees. For example, the third number of degrees maybe equal to or greater than 70 degrees, for example, 75 degrees, 80degrees, 85 degrees, 90 degrees, >90 degrees, or any suitable number ofdegrees. Alternatively, the third number of degrees may be less than 70degrees, for example, 65 degrees, 60 degrees, 55 degrees, <55 degrees,or any suitable number of degrees. The visual field may be the portionsof a screen of a head-mounted display that the user can see. The visualfield may vary from user to user and may depend on visual impairmentsspecific to each user.

A foreground area of a user's visual field may be any portion of theuser's visual field that the user can see with normal vision. Theforeground area may encompass a subset of the visual field. Theforeground area may encompass areas of the visual field that aresubstantially in the center of the user's visual field. The foregroundarea may be determined based on movements of the user's head. As anotherexample, the foreground area may be determined based on movements of theuser's center of gaze. For example, the foreground area may be within afourth number of degrees to the right and left of the center of gaze ofthe user, a fifth number of degrees above the center of gaze of theuser, and a sixth number of degrees below the center of gaze of theuser. For example, the fourth, fifth, and sixth numbers of degrees maybe equal to or greater than 18 degrees, for example, 20 degrees, 25degrees, 30 degrees, 35 degrees, >35 degrees, or any suitable number ofdegrees. Alternatively, the fourth, fifth, and sixth numbers of degreesmay be less than 18 degrees, for example, 15 degrees, 10 degrees, 5degrees, <5 degrees, or any suitable number of degrees. The foregroundarea may be the portions of a screen of a head-mounted display that theuser can see with main vision. The foreground area may vary from user touser and may depend on visual impairments specific to each user.

The control circuitry is further configured to generate for display, ina second portion of the virtual reality environment in the heads updisplay, additional content, wherein the second portion corresponds to aperipheral area of the visual field of the user.

The second portion of the virtual reality environment may be any portionof the virtual reality environment. The second portion may be a subsetof the virtual reality environment. The second portion may be differentthan the first portion. The second portion may be a top, bottom, right,or left portion of the virtual reality environment. The second portionmay be near or at the boundary or corner of the virtual realityenvironment. The second portion may be approximately a majority orapproximately a minority of the virtual reality environment.

The additional content assists the user to perform the full bodymovement. For example, the additional content may be a video of theuser's physical surroundings. As another example, the additional contentmay be a map of the user's physical surroundings.

A peripheral area of a user's visual field may be any portion of auser's visual field that the user can see with peripheral vision. Theperipheral area may be areas that are substantially at the edges of theuser's visual field. The peripheral area may be determined based onmovements of the user's head. As another example, the peripheral fieldmay be determined based on movements of the user's center of gaze. Forexample, the peripheral area may be any portion of the visual field thatis not foreground area. For example, the peripheral area may encompassareas of the visual field within a certain number of degrees of theouter boundary of the visual field. The peripheral area may be theportions of a screen of a head-mounted display that the user can seewith peripheral vision. The peripheral area may vary from user to userand may depend on visual impairments specific to each user.

The control circuitry is further configured, in response to detecting afull body movement of the user, to generate for display the additionalcontent in the first portion of the virtual reality environment. Thesystems and methods further include a detection module configured todetect the full body movement of the user. For example, the detectionmodule may detect the full body movement by detecting an acceleration ofa portion of the body of the user. For example, the detection module maydetect a footstep. As another example, the detection module may detectthe full body movement by detecting a change in a video of the physicalsurroundings of the user captured with a camera physically coupled tothe user.

In some embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the head-mounted displayincludes a camera feeding a video of the user's physical surroundings tothe display, and the control circuitry is further configured, whengenerating for display the additional content, to generate for displaythe video of the user's physical surroundings.

In certain embodiments, the control circuitry is further configured,when generating for display the additional content, to generate fordisplay the additional content as a picture-in-picture.

In some embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the detection moduleincludes an accelerometer included in the head-mounted display, and thedetection module is further configured, when detecting the full bodymovement of the user, to detect, using the accelerometer, a firstfootstep taken by the user.

In certain embodiments, the detection module is further configured todetect, using the accelerometer, a second footstep taken by the user,and the control circuitry is further configured, in response to thedetecting of the second footstep, to enlarge the additional content.

In some embodiments, the detection module is further configured todetect, using the accelerometer, a second footstep taken by the user,and the control circuitry is further configured, in response to thedetecting of the second footstep, to perform at least one of decreasingan opacity of the main content and increasing an opacity of theadditional content.

In certain embodiments, a user interface is configured to receive aninput from the user, and the control circuitry is further configured, inresponse to the input, to remove the additional content from the headsup display.

In some embodiments, the detection module is further configured, whendetecting the full body movement of the user, to detect a change in thevideo of the user's physical surroundings.

In certain embodiments, the detection module is further configured todetect that the user is substantially stationary, and the controlcircuitry is further configured, in response to the detecting that theuser is substantially stationary, to generate for display the additionalcontent in a third portion of the display.

In some embodiments, a user interface is configured to present an optionto the user to stop playback of the main content.

It should be noted that the systems and/or methods described above maybe applied to, or used in accordance with, other systems, methods and/orapparatuses.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIGS. 1-2 show an illustrative example of presenting additional contentin virtual reality environments on heads up displays showing maincontent without interfering with a user's viewing of the main content;

FIGS. 3-4 show an illustrative example of enabling a user to perform afull body movement while viewing a virtual reality environment on aheads up display without interfering with viewing of content on theheads up display;

FIG. 5 shows an illustrative head-mounted display for use as a heads updisplay in accordance with certain embodiments of the disclosure;

FIG. 6 shows an illustrative media listing display in accordance withsome embodiments of the disclosure;

FIG. 7 shows another illustrative media listing display in accordancewith some embodiments of the disclosure;

FIG. 8 is a block diagram of an illustrative user equipment device inaccordance with some embodiments of the disclosure;

FIG. 9 is a block diagram of an illustrative media system in accordancewith some embodiments of the disclosure;

FIG. 10 is an illustrative example of one component of a detectionmodule, which may be accessed in accordance with some embodiments of thedisclosure;

FIG. 11 is another illustrative example of one component of a detectionmodule, which may be accessed in accordance with some embodiments of thedisclosure;

FIG. 12 is another illustrative example of one component of a detectionmodule, which may be accessed in accordance with some embodiments of thedisclosure;

FIG. 13 is a flowchart of illustrative steps for presenting additionalcontent in virtual reality environments on heads up displays showingmain content without interfering with a user's viewing of the maincontent;

FIG. 14 is a flowchart of illustrative steps for enabling users toperform full body movements while viewing virtual reality environmentson heads up displays without interfering with viewing of content on theheads up displays;

FIG. 15 describes a process implemented on control circuitry to presentadditional content in virtual reality environments on heads up displaysshowing main content without interfering with a user's viewing of themain content in accordance with some embodiments of the disclosure; and

FIG. 16 describes a process to present additional content in virtualreality environments on heads up displays showing main content withoutinterfering with a user's viewing of the main content in accordance withsome embodiments of the disclosure.

DETAILED DESCRIPTION OF THE DRAWINGS

Methods and systems are described herein for presenting additionalcontent in virtual reality environments on heads up displays showingmain content without interfering with a user's viewing of the maincontent. Specifically, a movement of a center of gaze of the user isdetected. A visual field of the user based on the movement of the centerof gaze is determined. If it is determined that the portion of thevirtual reality environment in which the additional content is generatedfor display corresponds to a foreground area of the user's visual field,the additional content is generated for display in a portion of thevirtual reality environment corresponding to a peripheral area of theuser's visual field. By ensuring that additional content is generatedfor display in a portion of the virtual reality environment thatcorresponds to a peripheral area of the visual field of the user, it ispossible to ensure that the additional content does not interfere withthe user's viewing of main content corresponding to foreground areas ofthe visual field of the user.

Additionally, methods and systems are described herein for enabling auser to perform a full body movement while viewing a virtual realityenvironment on a head up displays without interfering with viewing ofcontent on the heads up display. Specifically, a full body movement ofthe user is detected. In response to detecting the full body movement,the additional content is generated for display in a portion of thevirtual reality environment corresponding to a foreground area of theuser's visual field. The additional content assists the user to performthe full body movement. By generating for display the additional contentin a portion of the virtual reality environment that corresponds to aforeground area of the user's visual field, it is possible to ensurethat the user can easily see the additional content. The additionalcontent is configured to assist the user in performing the full bodymovement without interfering with the user's viewing of main content onthe heads up display. For example, if the user is walking, theadditional content may be a video of the physical surroundings of theuser that can assist the user in, for example, avoiding obstacles in thephysical surroundings without needing to stop viewing content on theheads up display. Thus, generating for display the additional content inthe foreground area enables the user to perform the full body movementwithout interfering with the user's viewing of content on the heads updisplay.

As referred to herein, the term “virtual reality environment” should beunderstood to mean any non-physical content displayed to a user in sucha way that the non-physical content appears to the user to have asemblance of physicality. For example, the virtual reality environmentmay be a virtual world (for example, a virtual world in a game) whichappears to the user to be the world in which user is located. As anotherexample, the virtual reality environment may be non-physical contentthat appears to the user to be superimposed on the physical world. Forexample, the virtual reality environment may be a speedometer display(or any other display) that is superimposed on what user sees throughthe windshield of his or her car (or any other transparent surface). Asanother example, the virtual reality environment may be a media asset(for example, a television show or a movie) presented to the user suchthat the display of the media asset fully encompasses the visual fieldof the user.

As referred to herein, the term “heads up display” should be understoodto mean any display capable of displaying non-physical content to a userin such a way that the non-physical content appears to the user to havea semblance of physicality. For example, the heads up display may be ahead-mounted display that fully covers the eyes of the user. Thehead-mounted display may be configured as eyeglasses, binoculars, ahelmet, etc. As another example, the heads up display may be a display(for example, a display integrated with a windshield or eyeglasses) thatsuperimposes non-physical content on a view of the physical world whichthe user can see through the heads up display. As another example, theheads up display may be a room in which the user is located, where thewalls of the room are covered in display screens.

As referred to herein, the term “center of gaze” should be understood tomean a central portion of what a user can see. The center of gaze may beany area of the user's visual field to which the gaze of the user issubstantially focused. The center of gaze may be a midpoint between thegaze point of each eye of the user. In users with a visual impairment inone eye, the center of gaze may be the gaze point of the unimpaired eye.

As referred to herein, the term “visual field” should be understood tomean anything that a user can see when the user is in a specificposition. For example, the visual field may be determined based onmovements of the user's head. As another example, the visual field maybe determined based on movements of the user's center of gaze. Forexample, the visual field of the user may encompass areas within a firstnumber of degrees to the right and left of the user's center of gaze, asecond number of degrees above the center of gaze, and a third number ofdegrees below the center of gaze. For example, the first number ofdegrees may equal to or greater than 95 degrees, for example, 95degrees, 100 degrees, 105 degrees, 110 degrees, 115 degrees, 120degrees, >120 degrees, or any suitable number of degrees. Alternatively,the first number of degrees may be less than 95 degrees, for example, 90degrees, 85 degrees, 80 degrees, 75 degrees, 70 degrees, <70 degrees, orany suitable number of degrees. For example, the second number ofdegrees may be equal to or greater than 60 degrees, for example, 65degrees, 70 degrees, 75 degrees, 80 degrees, 85 degrees, >85 degrees, orany suitable number of degrees. Alternatively, the second number ofdegrees may be less than 60 degrees, for example, 55 degrees, 50degrees, 45 degrees, 40 degrees, <40 degrees, or any suitable number ofdegrees. For example, the third number of degrees may be equal to orgreater than 70 degrees, for example, 75 degrees, 80 degrees, 85degrees, 90 degrees, >90 degrees, or any suitable number of degrees.Alternatively, the third number of degrees may be less than 70 degrees,for example, 65 degrees, 60 degrees, 55 degrees, <55 degrees, or anysuitable number of degrees. The visual field may be the portions of ascreen of a head-mounted display that the user can see. The visual fieldof the user may vary from user to user and may depend on visualimpairments specific to each user.

As referred to herein, the term “foreground area” should be understoodto mean any portion of the user's visual field that the user can seewith normal vision. The foreground area may encompass a subset of thevisual field. The foreground area may encompass areas of the visualfield that are substantially in the center of the user's visual field.The foreground area may be determined based on movements of the user'shead. As another example, the foreground area may be determined based onmovements of the user's center of gaze. For example, the foreground areamay be within a fourth number of degrees to the right and left of thecenter of gaze of the user, a fifth number of degrees above the centerof gaze of the user, and a sixth number of degrees below the center ofgaze of the user. For example, the fourth, fifth, and sixth numbers ofdegrees may be equal to or greater than 18 degrees, for example, 20degrees, 25 degrees, 30 degrees, 35 degrees, >35 degrees, or anysuitable number of degrees. Alternatively, the fourth, fifth, and sixthnumbers of degrees may be less than 18 degrees, for example, 15 degrees,10 degrees, 5 degrees, <5 degrees, or any suitable number of degrees.The foreground area may be the portions of a screen of a head-mounteddisplay that the user can see with main vision. The foreground area mayvary from user to user and may depend on visual impairments specific toeach user.

As referred to herein, the term “peripheral area” should be understoodto mean any portion of a user's visual field that the user can see withperipheral vision. The peripheral area may be areas that aresubstantially at the edges of the user's visual field. The peripheralarea may be determined based on movements of the user's head. As anotherexample, the peripheral field may be determined based on movements ofthe user's center of gaze. For example, the peripheral area may be anyportion of the visual field that is not foreground area. For example,the peripheral area may encompass areas of the visual field within acertain number of degrees of the outer boundary of the visual field. Theperipheral area may be the portions of a screen of a head-mounteddisplay that the user can see with peripheral vision. The peripheralarea may vary from user to user and may depend on visual impairmentsspecific to each user.

As referred to herein, the term “main content” should be understood tomean any content that is intended to be the object of a user's mainfocus. For example, the main content may be a media asset, such as amovie, a television show, a video game, or a virtual reality world. Asanother example, the main content may be a media guidance screen.

As referred to herein, the term “additional content” should beunderstood to mean any content that is not main content. The additionalcontent may be unrelated to the main content or related to the maincontent. For example, the additional content may be a video of theuser's physical surroundings, stock price information, sports scoreinformation, news information, weather information, a clock, or aschedule of events.

As referred to herein, the term “full body movement” should beunderstood to mean any physical movement by a user that requiresmovement of a substantial portion of the user's entire body. Forexample, the full body movement may be walking, jumping, standing up,sitting down, rotating one's body, etc.

The amount of content available to users in any given content deliverysystem can be substantial. Consequently, many users desire a form ofmedia guidance through an interface that allows users to efficientlynavigate content selections and easily identify content that they maydesire. An application that provides such guidance is referred to hereinas an interactive media guidance application or, sometimes, a mediaguidance application or a guidance application.

Interactive media guidance applications may take various forms dependingon the content for which they provide guidance. One typical type ofmedia guidance application is an interactive television program guide.Interactive television program guides (sometimes referred to aselectronic program guides) are well-known guidance applications that,among other things, allow users to navigate among and locate many typesof content or media assets. Interactive media guidance applications maygenerate graphical user interface screens that enable a user to navigateamong, locate and select content. As referred to herein, the terms“media asset” and “content” should be understood to mean anelectronically consumable user asset, such as television programming, aswell as pay-per-view programs, on-demand programs (as in video-on-demand(VOD) systems), Internet content (e.g., streaming content, downloadablecontent, Webcasts, etc.), video clips, audio, content information,pictures, rotating images, documents, playlists, websites, articles,books, electronic books, blogs, advertisements, chat sessions, socialmedia, applications, games, and/or any other media or multimedia and/orcombination of the same. Guidance applications also allow users tonavigate among and locate content. As referred to herein, the term“multimedia” should be understood to mean content that utilizes at leasttwo different content forms described above, for example, text, audio,images, video, or interactivity content forms. Content may be recorded,played, displayed or accessed by user equipment devices, but can also bepart of a live performance.

The media guidance application and/or any instructions for performingany of the embodiments discussed herein may be encoded on computerreadable media. Computer readable media includes any media capable ofstoring data. The computer readable media may be transitory, including,but not limited to, propagating electrical or electromagnetic signals,or may be non-transitory including, but not limited to, volatile andnon-volatile computer memory or storage devices such as a hard disk,floppy disk, USB drive, DVD, CD, media cards, register memory, processorcaches, Random Access Memory (“RAM”), etc.

With the advent of the Internet, mobile computing, and high-speedwireless networks, users are accessing media on user equipment deviceson which they traditionally did not. As referred to herein, the phrase“user equipment device,” “user equipment,” “user device,” “electronicdevice,” “electronic equipment,” “media equipment device,” or “mediadevice” should be understood to mean any device for accessing thecontent described above, such as a television, a Smart TV, a set-topbox, an integrated receiver decoder (IRD) for handling satellitetelevision, a digital storage device, a digital media receiver (DMR), adigital media adapter (DMA), a streaming media device, a DVD player, aDVD recorder, a connected DVD, a local media server, a BLU-RAY player, aBLU-RAY recorder, a personal computer (PC), a laptop computer, a tabletcomputer, a WebTV box, a personal computer television (PC/TV), a PCmedia server, a PC media center, a hand-held computer, a stationarytelephone, a personal digital assistant (PDA), a mobile telephone, aportable video player, a portable music player, a portable gamingmachine, a smart phone, or any other television equipment, computingequipment, or wireless device, and/or combination of the same. In someembodiments, the user equipment device may have a front facing screenand a rear facing screen, multiple front screens, or multiple angledscreens. In some embodiments, the user equipment device may have a frontfacing camera and/or a rear facing camera. On these user equipmentdevices, users may be able to navigate among and locate the same contentavailable through a television. Consequently, media guidance may beavailable on these devices, as well. The guidance provided may be forcontent available only through a television, for content available onlythrough one or more of other types of user equipment devices, or forcontent available both through a television and one or more of the othertypes of user equipment devices. The media guidance applications may beprovided as on-line applications (i.e., provided on a web-site), or asstand-alone applications or clients on user equipment devices. Variousdevices and platforms that may implement media guidance applications aredescribed in more detail below.

One of the functions of the media guidance application is to providemedia guidance data to users. As referred to herein, the phrase “mediaguidance data” or “guidance data” should be understood to mean any datarelated to content or data used in operating the guidance application.For example, the guidance data may include program information, guidanceapplication settings, user preferences, user profile information, medialistings, media-related information (e.g., broadcast times, broadcastchannels, titles, descriptions, ratings information (e.g., parentalcontrol ratings, critic's ratings, etc.), genre or category information,actor information, logo data for broadcasters' or providers' logos,etc.), media format (e.g., standard definition, high definition, 3D,etc.), advertisement information (e.g., text, images, media clips,etc.), on-demand information, blogs, websites, and any other type ofguidance data that is helpful for a user to navigate among and locatedesired content selections.

In some embodiments, control circuitry 604, discussed further inrelation to FIG. 8 below, executes instructions stored in memory (i.e.,storage 608 discussed further in relation to FIG. 8 below).Specifically, control circuitry 604 may be instructed to perform thefunctions discussed above and below. For example, the instructions maycause control circuitry 604 to generate the displays as described aboveand below. In some implementations, any action performed by controlcircuitry 604 may be based on instructions.

FIGS. 1-2 show an illustrative example of presenting additional contentin virtual reality environments on heads up displays showing maincontent without interfering with a user's viewing of the main content.

Virtual reality environment 104 is any non-physical content displayed touser 124 in such a way that the non-physical content appears to user 124to have a semblance of physicality. For example, virtual realityenvironment 104 may be a virtual world (for example, a virtual world ina game) which appears to user 124 to be the world in which user 124 islocated. As another example, virtual reality environment 104 may benon-physical content that appears to user 124 to be superimposed on thephysical world. For example, the virtual reality environment 104 may bea speedometer display that is superimposed on what user 124 sees throughthe windshield of his or her car. As another example, virtual realityenvironment 104 may be a media asset (for example, a television show ora movie) presented to user 124 such that the display of the media assetfully encompasses the visual field of user 124.

Heads up display 102 generates for display virtual reality environment104 to user 124. Heads up display 102 may be any display capable ofdisplaying non-physical content to user 124 in such a way that thenon-physical content appears to user 124 to have a semblance ofphysicality. For example, heads up display 102 may be a head-mounteddisplay that fully covers the eyes of user 124 (for example,head-mounted display 301 of FIG. 5). As another example, heads updisplay 102 may be a display (for example, a display integrated with awindshield or eyeglasses) that superimposes non-physical content on aview of the physical world which user 124 can see through heads updisplay 102. Heads up display 102 may be configured to change thecontent of virtual reality environment 104 shown to user 124 when user124 moves. For example, if user 124 walks forward, the content ofvirtual reality environment 104 shown to user 124 may appear to the userto move closer to user 124.

User 124 has first visual field 114. First visual field 114 encompasseswhat user 124 can see when user 124 is in the position shown in FIG. 1.For simplicity, in FIGS. 1-4 and this description, visual fields areshown and discussed as varying in the horizontal direction, but itshould be understood that visual fields also vary in the verticaldirection. The center of gaze of a user is the central portion of whatthe user can see. In FIG. 1, the center of gaze of user 124 is at centerof gaze position 128. A visual field of a user extends to the right andleft of the center of gaze position a certain number of degrees (forexample, 95 degrees). The visual field includes at least one foregroundarea and at least one peripheral area. The at least one foreground areaincludes the area of the visual field that the user can see with normalvision. The at least one foreground area encompasses a certain number ofdegrees (for example, 60 degrees) of the center of the visual field. Theat least one peripheral area includes the area of the visual field thatthe user can see with peripheral vision. The at least one peripheralarea encompasses a certain number of degrees (for example, 35 degrees)to the right and left of foreground area 116. First visual field 114includes foreground area 116, peripheral area 118, and peripheral area120.

In FIG. 1, heads up display 102 shows virtual reality environmentportion 142 of virtual reality environment 104. Virtual realityenvironment portion 142 of virtual reality environment 104 includesfirst portion 108 and second portion 110. First portion 108 includesportions of virtual reality environment 104 corresponding to foregroundarea 116 of first visual field 114. Second portion 110 includes portionsof virtual reality environment 104 corresponding to peripheral area 118of first visual field 114. First main content 106 (in FIG. 1, a tree) isgenerated for display in first portion 108 of virtual realityenvironment 104. Second main content 144 (in FIGS. 1-2, a cloud) andadditional content 122 (in FIGS. 1-2, a video of the user's physicalsurroundings) are generated for display in second portion 110.

In FIG. 2, user 124 has rotated such that the center of gaze of user 124is at center of gaze position 138. User 124 has second visual field 126when user 124 is in the position shown in FIG. 2. Second visual field126 includes foreground area 130, peripheral area 132, and peripheralarea 134.

In FIG. 2, heads up display 102 shows virtual reality environmentportion 140 of virtual reality environment 104 to user 124. Virtualreality environment portion 140 of virtual reality environment 104includes second portion 110 and third portion 136. Second portion 110includes portions of virtual reality environment 104 corresponding toforeground area 130 of second visual field 126. Third portion 136includes portions of virtual reality environment 104 corresponding toperipheral area 132 of second visual field 126.

Because user 124 has rotated to the left from FIG. 1 to FIG. 2, virtualreality environment portion 140 of virtual reality environment 104includes portions to the left of virtual reality environment portion 142of virtual environment 104 but also portions that are present in bothvirtual reality environment portion 140 and virtual reality environmentportion 142. For example, second portion 110 is present both in virtualreality environment portion 140 and virtual reality environment portion142, but third portion 136 is present in virtual reality environmentportion 140 but not in virtual reality environment portion 142.

Movements of the center of gaze of user 124 may be detected (forexample, by detection module 616 of FIG. 8). In some embodiments,detecting the movement of the center of gaze includes detecting thecenter of the gaze. For example, in FIGS. 1-2, the movement of thecenter of gaze of user 124 from center of gaze position 128 to center ofgaze position 138 is detected. A visual field of user 124 based on themovement of the center of gaze of user 124 is determined. For example,in FIG. 2, the second visual field 126 is determined based on themovement of the center of gaze of user 124 from center of gaze position128 to center of gaze position 138.

Before the movement of the center of gaze of user 124, additionalcontent 122 was generated for display in a portion (i.e., second portion110) of virtual reality environment 104 that corresponded to aperipheral area (i.e., peripheral area 118) of the visual field (i.e.,first visual field 114) of user 124. A determination is made if theportion (i.e., second portion 110) of virtual reality environment 104 inwhich additional content 122 was generated for display before themovement of the center of gaze of user 124 corresponds to the foregroundarea (i.e., foreground area 130) of the visual field (i.e., secondvisual field 126) of user 124 after the movement of the center of gazeof user 124. For example, in FIG. 2, second portion 110 corresponds toforeground area 130 of second visual field 126. In response to thisdetermination, additional content 122 is generated for display in aportion of the visual field of user 124 that corresponds to a peripheralarea of the visual field (i.e., second visual field 126) of user 124.For example, in FIG. 2, additional content 122 is generated for displayin third portion 136, which corresponds to peripheral area 132 of secondvisual field 126.

By ensuring that additional content 122 is generated for display in aportion of virtual reality environment 104 that corresponds to aperipheral area of the visual field of user 124, it is possible toensure that additional content 122 does not interfere with the user'sviewing of main content corresponding to foreground areas of the visualfield of user 124. For example, in FIGS. 1-2, if additional content 122remained generated for display in second portion 110 of virtual realityenvironment 104 after the movement of the center of gaze of user 124,additional content 122 would interfere with the user's viewing of secondmain content 144 in foreground area 130. Because additional content 122is generated for display in third portion 136 after the movement of thecenter of gaze of user 124, additional content 122 does not interferewith the user's viewing of second main content 144 in foreground area130.

First main content 106 and second main content 144 may be any type ofcontent. In some embodiments, main content 106 provides media guidancedata (as discussed in relation to FIGS. 6-7).

Additional content 122 may be any type of additional content. Forexample, additional content 122 may be a video of the physicalsurroundings of user 124, stock price information, sports scoreinformation, news information, weather information, a clock, a scheduleof events, or any other type of additional content. Additional content122 may be unrelated to main content 106.

In FIGS. 1-2, additional content 122 is generated for display as apicture-in-picture. In some embodiments, additional content 122 isgenerated for display as an overlay.

In some embodiments, additional content 122 is generated for display ata lower image and/or video quality (e.g., lower resolution, frame rate,etc.) than main content (e.g., first main content 106, second maincontent 144) shown by virtual reality environment 104. Generating fordisplay additional content 122 at a lower image and/or video quality mayhelp heads up display 102 conserve power, memory, bandwidth, etc.Additionally, generating for display additional content 122 at a lowerimage and/or video quality may not be detrimental to the user's viewingexperience because user 124 may not be able to differentiate betweenhigh and low quality images and/or video viewed in peripheral areas(e.g., peripheral area 118) of the user's visual field (e.g., visualfield 114).

In some embodiments, virtual reality environment 104 includes portionsthat show main content and portions that do not show main content. Forexample, a certain portion of the center of virtual reality environment104 may show main content (e.g., first main content 106, second maincontent 144) while the remainder of virtual reality environment 104 doesnot show main content. For example, virtual reality environment 104 mayshow main content in a certain portion (e.g., 95%, 90%, 80%, 70%, 60%,50%, etc.) of the center of the horizontal extent of virtual realityenvironment 104 but not show main content in the remainder of thehorizontal extent of virtual reality environment 104. As anotherexample, virtual reality environment 104 may show main content in acertain portion (e.g., 95%, 90%, 80%, 70%, 60%, 50%, etc.) of the centerof the vertical extent of virtual reality environment 104 but not showmain content in the remainder of the vertical extent of virtual realityenvironment 104. As another example, virtual reality environment 104 maynot show main content in both a certain portion of the horizontal extentand the vertical extent of virtual reality environment 104. Theremainder of virtual reality environment 104 in which main content isnot shown may be blank space and may correspond to a peripheral area(e.g., peripheral areas 118, 120, 132, and 134) of the user's visualfield (e.g., visual fields 114 and 126). Additional content 122 may begenerated for display in the portion of virtual reality environment 104that does not show main content. Thus, additional content 122 does notinterfere with the main content even when additional content 122 isgenerated for display in a portion of virtual reality environment 104corresponding to a peripheral area of the user's visual field.

FIGS. 3-4 show an illustrative example of enabling a user to perform afull body movement while viewing a virtual reality environment on aheads up display without interfering with viewing of content on theheads up display.

In FIGS. 3-4, heads up display 202 shows virtual reality environmentportion 242 of virtual reality environment 204. Virtual realityenvironment portion 242 of virtual reality environment 204 includesfirst portion 208 and second portion 210. First portion 208 includesportions of virtual reality environment 204 corresponding to foregroundarea 216 of visual field 214. Second portion 210 includes portions ofvirtual reality environment 204 corresponding to peripheral area 218 ofvisual field 214. Main content 206 is generated for display in firstportion 208 of virtual reality environment 204. Additional content 222is generated for display in second portion 210 of virtual realityenvironment 204. In FIG. 3, user 224 is sitting. In FIG. 4, user 224 iswalking.

Full body movements of user 224 may be detected (for example, bydetection module 616 on FIG. 8). For example, in FIGS. 3-4, the changeof user 224 from a sitting position to a walking position is detected.In response to detecting the full body movement of user, additionalcontent 222 is generated for display in first portion 208 of virtualreality environment 204. Additional content 222 may be any content thatassists user 224 in performing the full body movement. For example, inFIGS. 3-4, additional content 222 is a video of the physicalsurroundings of user 224. Alternatively, additional content 222 may be amap of the location of user 224, etc.

By generating for display additional content 222 in a portion of virtualreality environment 204 that corresponds to foreground area 216, it ispossible to ensure that user 224 can easily see additional content 222.Additional content 222 is configured to assist user 224 in performingthe full body movement without interfering with the user's viewing ofmain content on heads up display 202. For example, if user 224 iswalking, additional content 222 may be a video of the physicalsurroundings of user 224 that can assist the user 224 in, for example,avoiding obstacles in the physical surroundings without needing to stopviewing content on heads up display 202. Thus, generating for displayadditional content 222 in foreground area 216 enables the user 224 toperform the full body movement without interfering with the user'sviewing of content on heads up display 202.

Main content 206 may be any type of content. In some embodiments, maincontent 206 provides media guidance data (as discussed in relation toFIGS. 6-7).

In FIGS. 3-4, additional content 222 is generated for display as apicture-in-picture. In some embodiments, additional content 222 isgenerated for display as an overlay.

In some embodiments, when additional content 222 is generated fordisplay in a portion (e.g., second portion 210) of virtual realityenvironment 204 corresponding to a peripheral area (e.g., peripheralarea 218) of the visual field (e.g., visual field 214) of user 224, theadditional content 222 is generated for display at a lower image and/orvideo quality (e.g., lower resolution, frame rate, etc.) than maincontent (e.g., main content 206) shown by virtual reality environment204. Generating for display additional content 222 at a lower imageand/or video quality may help heads up display 202 conserve power,memory, bandwidth, etc. Additionally, generating for display additionalcontent 222 at a lower image and/or video quality may not be detrimentalto the user's viewing experience because user 224 may not be able todifferentiate between high and low quality images and/or video viewed inperipheral areas (e.g., peripheral area 218) of the user's visual field(e.g., visual field 214). When additional content 222 is generated fordisplay in a portion (e.g., first portion 208) of virtual realityenvironment 204 corresponding to a foreground area (e.g., foregroundarea 216) of the user's visual field (e.g., visual field 214), the imageand/or video quality of the additional content 222 may be increased.User 224 may be able to differentiate between high and low qualityimages and/or video viewed in foreground areas of the user's visualfield. By generating for display additional content 222 at a higherimage and/or video quality, it is ensured that the user's viewing ofadditional content 222 is not compromised (e.g., by user 224 needing toview a low quality version of additional content 222).

In some embodiments, virtual reality environment 204 includes portionsthat show main content and portions that do not show main content. Forexample, a certain portion of the center of virtual reality environment204 may show main content (e.g., main content 206) while the remainderof virtual reality environment 204 does not show main content. Forexample, virtual reality environment 204 may show main content in acertain portion (e.g., 95%, 90%, 80%, 70%, 60%, 50%, etc.) of the centerof the horizontal extent of virtual reality environment 204 but not showmain content in the remainder of the horizontal extent of virtualreality environment 204. As another example, virtual reality environment204 may show main content in a certain portion (e.g., 95%, 90%, 80%,70%, 60%, 50%, etc.) of the center of the vertical extent of virtualreality environment 204 but not show main content in the remainder ofthe vertical extent of virtual reality environment 204. As anotherexample, virtual reality environment 204 may not show main content inboth a certain portion of the horizontal extent and the vertical extentof virtual reality environment 204. The remainder of virtual realityenvironment 204 in which main content is not shown may be blank spaceand may correspond to a peripheral area (e.g., peripheral areas 218 and220) of the user's visual field (e.g., visual field 214). Before thefull body movement of the user is detected, additional content 222 maybe generated for display in a portion of virtual reality environment 204that does not show main content. Thus, additional content 222 does notinterfere with the main content even when additional content 222 isgenerated for display in a portion of virtual reality environment 204corresponding to a peripheral area of the user's visual field. After thefull body movement of the user is detected, additional content 222 maybe generated for display in a portion of the virtual reality environment204 that does show main content.

FIG. 5 shows an illustrative head-mounted display for use as a heads updisplay in accordance with certain embodiments of the disclosure.Head-mounted display 301 includes headset 303 and mobile phone 305.Headset 303 includes display 307, straps 309 and 311, and clips 313 and315. Mobile phone 305 includes camera 317. Head-mounted display 301 isphysically coupled to the head of user 319.

Display 307 is located over the eyes of user 319. Because display 307completely covers the eyes of user 319, display 307 gives content itgenerates for display a semblance of reality for user 319. Therefore,display 307 operates as a heads up display. Display 307 is physicallycoupled to straps 309 and 311. Strap 309 wraps around the head of user319 and strap 311 wraps over the head of user 319. Straps 309 and 311secure display 307 over the eyes of user 317. Other configurations forheadset 303 are possible. For example, one or both of straps 309 and 311may be absent. In some embodiments, headset 303 is configured as ahelmet sitting on the head of user 319. In some embodiments, headset 303is configured as eyeglasses resting on the ears of user 319. In someembodiments, display 307 is located over only one eye of user 319.

Mobile phone 305 is physically coupled to headset 303 such that mobilephone 305 is located on the opposite side of display 307 as the eyes ofuser 317. Mobile phone 305 is physically coupled to headset 303 by clips313 and 315. Other configurations for mobile phone 305 are possible. Forexample, one or both of clips 313 and 315 may be absent. Mobile phone305 may be physically coupled to headset 303 by fitting snugly into arecess in headset 303. Mobile phone 305 may be physically coupled toheadset 303 by straps, screws, adhesive, or any other coupling means. Apanel or lip physically coupled to headset 303 may hold mobile phone 305in place. Mobile phone 305 may be inserted into a slot in headset 303.

Camera 317 faces away from display 307 such that camera 317 is capableof capturing images and video of the physical surroundings of user 317.Camera 317 may feed a video of the physical surroundings of user 317 tohead-mounted display 301. The video of the physical surroundings of user317 may be used as additional content 122 or 222. In some embodiments,the video of the physical surroundings of user 317 is fed tohead-mounted display 301 by an electrical connection betweenhead-mounted display 301 and mobile phone 305. For example, there may bean electrical cable with a USB interface serving as the electricalconnection. In some embodiments, the video of the physical surroundingsof user 317 may be fed to head-mounted display 301 wirelessly, such asover a wireless network or through a Bluetooth connection. In someembodiments, a screen on mobile phone 305 displays the video of thephysical surroundings of user 317, and the screen on mobile phone 305 isvisible through display 307 of head-mounted display 301.

In some embodiments, mobile phone 305 is absent, and camera 317 isintegrated into headset 303. In some embodiments, mobile phone 305 isreplaced by another electronic device, such as a tablet device.

Head-mounted display 301 may provide user 319 with a more immersiveviewing experience than other devices provide. For example, because thedisplay 307 fully covers the user's eyes, content displayed by display307 may have a semblance of reality for user 319. Additionally, user 319user may view content on the display 307 without distractions from thephysical surroundings.

However, challenges arise because the display 307 fully covers the eyesof user 319. For example, user 319 may wish to monitor something in thephysical surroundings, such as a child, food being cooked, peopleentering the room in which the user 319 is located, etc. Monitoring thephysical surroundings by user 319 is difficult if the display 307 fullycover's the eyes of user 319. Therefore, displaying on display 307additional content (for example, additional content 122) consisting of avideo feed of the user's physical surroundings captured by the camera317 may be desirable.

However, challenges arise with displaying a video of the user's physicalsurroundings as additional content on the display 307. The video of theuser's physical surroundings may interfere with the user's viewing ofmain content (for example, second main content 144) on the display 307.In particular, if the video of the user's physical surroundings isdisplayed on a portion of the display 307 corresponding to a foregroundarea of the visual field of user 319, the video of the user's physicalsurroundings may interfere with the user's viewing of the main contenton the display 307. Therefore, in accordance with embodiments describedabove in relation to FIGS. 1-2, additional content 122 is generated fordisplay in a portion of a virtual reality environment corresponding to aperipheral area of the visual field of the user. If a change in thevisual field of the user (for example, due to a movement in the centerof gaze of the user) is detected, and it is determined that the portionof the virtual reality environment in which the additional content isgenerated for display corresponds to a foreground area of the visualfield of the user, the additional content is moved to a portion of thevirtual reality environment corresponding to a peripheral area of thevisual field of the user.

Further challenges arise if user 319 performs full body movements whilewearing head-mounted display 301. For example, user 319 may desire toview content on head-mounted display 301 while walking, but may beunable to see obstacles in the physical surroundings. Therefore, inaccordance with embodiments described above in relation to FIGS. 3-4,additional content 222 is generated for display in a portion of virtualreality environment 204 corresponding to a foreground area of the visualfield of the user. Additional content 222 is configured to assist theuser in performing the full body movement. For example, additionalcontent 222 may be a video of the user's physical surroundings. Bygenerating for display additional content 222 in a portion of virtualreality environment 204 corresponding to a foreground area of the visualfield of the user, the user is able to easily see the additional content222. The user can use additional content 222 to perform the full bodymovement and avoid having to remove head-mounted display 301 and havingto interfere with viewing of content on head-mounted display 301.

FIGS. 6-7 show illustrative display screens that may be used to providemedia guidance data. The display screens shown in FIGS. 6-7 may beimplemented on any suitable user equipment device or platform (forexample, heads up display 102 or head-mounted display 301). While thedisplays of FIGS. 6-7 are illustrated as full screen displays, they mayalso be fully or partially overlaid over content being displayed. A usermay indicate a desire to access content information by selecting aselectable option provided in a display screen (e.g., a menu option, alistings option, an icon, a hyperlink, etc.) or pressing a dedicatedbutton (e.g., a GUIDE button) on a remote control or other user inputinterface or device. In response to the user's indication, the mediaguidance application may provide a display screen with media guidancedata organized in one of several ways, such as by time and channel in agrid, by time, by channel, by source, by content type, by category(e.g., movies, sports, news, children, or other categories ofprogramming), or other predefined, user-defined, or other organizationcriteria.

FIG. 6 shows illustrative grid of a program listings display 400arranged by time and channel that also enables access to different typesof content in a single display. Display 400 may include grid 402 with:(1) a column of channel/content type identifiers 404, where eachchannel/content type identifier (which is a cell in the column)identifies a different channel or content type available; and (2) a rowof time identifiers 406, where each time identifier (which is a cell inthe row) identifies a time block of programming. Grid 402 also includescells of program listings, such as program listing 408, where eachlisting provides the title of the program provided on the listing'sassociated channel and time. With a user input device, a user can selectprogram listings by moving highlight region 410. Information relating tothe program listing selected by highlight region 410 may be provided inprogram information region 412. Region 412 may include, for example, theprogram title, the program description, the time the program is provided(if applicable), the channel the program is on (if applicable), theprogram's rating, and other desired information.

In addition to providing access to linear programming (e.g., contentthat is scheduled to be transmitted to a plurality of user equipmentdevices at a predetermined time and is provided according to aschedule), the media guidance application also provides access tonon-linear programming (e.g., content accessible to a user equipmentdevice at any time and is not provided according to a schedule).Non-linear programming may include content from different contentsources including on-demand content (e.g., VOD), Internet content (e.g.,streaming media, downloadable media, etc.), locally stored content(e.g., content stored on any user equipment device described above orother storage device), or other time-independent content. On-demandcontent may include movies or any other content provided by a particularcontent provider (e.g., HBO On Demand providing “The Sopranos” and “CurbYour Enthusiasm”). HBO ON DEMAND is a service mark owned by Time WarnerCompany L.P. et al. and THE SOPRANOS and CURB YOUR ENTHUSIASM aretrademarks owned by the Home Box Office, Inc. Internet content mayinclude web events, such as a chat session or Webcast, or contentavailable on-demand as streaming content or downloadable content throughan Internet web site or other Internet access (e.g. FTP).

Grid 402 may provide media guidance data for non-linear programmingincluding on-demand listing 414, recorded content listing 416, andInternet content listing 418. A display combining media guidance datafor content from different types of content sources is sometimesreferred to as a “mixed-media” display. Various permutations of thetypes of media guidance data that may be displayed that are differentthan display 400 may be based on user selection or guidance applicationdefinition (e.g., a display of only recorded and broadcast listings,only on-demand and broadcast listings, etc.). As illustrated, listings414, 416, and 418 are shown as spanning the entire time block displayedin grid 402 to indicate that selection of these listings may provideaccess to a display dedicated to on-demand listings, recorded listings,or Internet listings, respectively. In some embodiments, listings forthese content types may be included directly in grid 402. Additionalmedia guidance data may be displayed in response to the user selectingone of the navigational icons 420. (Pressing an arrow key on a userinput device may affect the display in a similar manner as selectingnavigational icons 420.)

Display 400 may also include video region 422, advertisement 424, andoptions region 426. Video region 422 may allow the user to view and/orpreview programs that are currently available, will be available, orwere available to the user. The content of video region 422 maycorrespond to, or be independent from, one of the listings displayed ingrid 402. Grid displays including a video region are sometimes referredto as picture-in-guide (PIG) displays. PIG displays and theirfunctionalities are described in greater detail in Satterfield et al.U.S. Pat. No. 6,564,378, issued May 13, 2003 and Yuen et al. U.S. Pat.No. 6,239,794, issued May 29, 2001, which are hereby incorporated byreference herein in their entireties. PIG displays may be included inother media guidance application display screens of the embodimentsdescribed herein.

Advertisement 424 may provide an advertisement for content that,depending on a viewer's access rights (e.g., for subscriptionprogramming), is currently available for viewing, will be available forviewing in the future, or may never become available for viewing, andmay correspond to or be unrelated to one or more of the content listingsin grid 402. Advertisement 424 may also be for products or servicesrelated or unrelated to the content displayed in grid 402. Advertisement424 may be selectable and provide further information about content,provide information about a product or a service, enable purchasing ofcontent, a product, or a service, provide content relating to theadvertisement, etc. Advertisement 424 may be targeted based on a user'sprofile/preferences, monitored user activity, the type of displayprovided, or on other suitable targeted advertisement bases.

While advertisement 424 is shown as rectangular or banner shaped,advertisements may be provided in any suitable size, shape, and locationin a guidance application display. For example, advertisement 424 may beprovided as a rectangular shape that is horizontally adjacent to grid402. This is sometimes referred to as a panel advertisement. Inaddition, advertisements may be overlaid over content or a guidanceapplication display or embedded within a display. Advertisements mayalso include text, images, rotating images, video clips, or other typesof content described above. Advertisements may be stored in a userequipment device having a guidance application, in a database connectedto the user equipment, in a remote location (including streaming mediaservers), or on other storage means, or a combination of theselocations. Providing advertisements in a media guidance application isdiscussed in greater detail in, for example, Knudson et al., U.S. PatentApplication Publication No. 2003/0110499, filed Jan. 17, 2003; Ward, IIIet al. U.S. Pat. No. 6,756,997, issued Jun. 29, 2004; and Schein et al.U.S. Pat. No. 6,388,714, issued May 14, 2002, which are herebyincorporated by reference herein in their entireties. It will beappreciated that advertisements may be included in other media guidanceapplication display screens of the embodiments described herein.

Options region 426 may allow the user to access different types ofcontent, media guidance application displays, and/or media guidanceapplication features. Options region 426 may be part of display 400 (andother display screens described herein), or may be invoked by a user byselecting an on-screen option or pressing a dedicated or assignablebutton on a user input device. The selectable options within optionsregion 426 may concern features related to program listings in grid 402or may include options available from a main menu display. Featuresrelated to program listings may include searching for other air times orways of receiving a program, recording a program, enabling seriesrecording of a program, setting program and/or channel as a favorite,purchasing a program, or other features. Options available from a mainmenu display may include search options, VOD options, parental controloptions, Internet options, cloud-based options, device synchronizationoptions, second screen device options, options to access various typesof media guidance data displays, options to subscribe to a premiumservice, options to edit a user's profile, options to access a browseoverlay, or other options.

The media guidance application may be personalized based on a user'spreferences. A personalized media guidance application allows a user tocustomize displays and features to create a personalized “experience”with the media guidance application. This personalized experience may becreated by allowing a user to input these customizations and/or by themedia guidance application monitoring user activity to determine varioususer preferences. Users may access their personalized guidanceapplication by logging in or otherwise identifying themselves to theguidance application. Customization of the media guidance applicationmay be made in accordance with a user profile. The customizations mayinclude varying presentation schemes (e.g., color scheme of displays,font size of text, etc.), aspects of content listings displayed (e.g.,only HDTV or only 3D programming, user-specified broadcast channelsbased on favorite channel selections, re-ordering the display ofchannels, recommended content, etc.), desired recording features (e.g.,recording or series recordings for particular users, recording quality,etc.), parental control settings, customized presentation of Internetcontent (e.g., presentation of social media content, e-mail,electronically delivered articles, etc.) and other desiredcustomizations.

The media guidance application may allow a user to provide user profileinformation or may automatically compile user profile information. Themedia guidance application may, for example, monitor the content theuser accesses and/or other interactions the user may have with theguidance application. Additionally, the media guidance application mayobtain all or part of other user profiles that are related to aparticular user (e.g., from other web sites on the Internet the useraccesses, such as www.allrovi.com, from other media guidanceapplications the user accesses, from other interactive applications theuser accesses, from another user equipment device of the user, etc.),and/or obtain information about the user from other sources that themedia guidance application may access. As a result, a user can beprovided with a unified guidance application experience across theuser's different user equipment devices. This type of user experience isdescribed in greater detail below in connection with FIG. 9. Additionalpersonalized media guidance application features are described ingreater detail in Ellis et al., U.S. Patent Application Publication No.2005/0251827, filed Jul. 11, 2005, Boyer et al., U.S. Pat. No.7,165,098, issued Jan. 16, 2007, and Ellis et al., U.S. PatentApplication Publication No. 2002/0174430, filed Feb. 21, 2002, which arehereby incorporated by reference herein in their entireties.

Another display arrangement for providing media guidance is shown inFIG. 7. Video mosaic display 500 includes selectable options 502 forcontent information organized based on content type, genre, and/or otherorganization criteria. In display 500, television listings option 504 isselected, thus providing listings 506, 508, 510, and 512 as broadcastprogram listings. In display 500 the listings may provide graphicalimages including cover art, still images from the content, video clippreviews, live video from the content, or other types of content thatindicate to a user the content being described by the media guidancedata in the listing. Each of the graphical listings may also beaccompanied by text to provide further information about the contentassociated with the listing. For example, listing 508 may include morethan one portion, including media portion 514 and text portion 516.Media portion 514 and/or text portion 516 may be selectable to viewcontent in full-screen or to view information related to the contentdisplayed in media portion 514 (e.g., to view listings for the channelthat the video is displayed on).

The listings in display 500 are of different sizes (i.e., listing 506 islarger than listings 508, 510, and 512), but if desired, all thelistings may be the same size. Listings may be of different sizes orgraphically accentuated to indicate degrees of interest to the user orto emphasize certain content, as desired by the content provider orbased on user preferences. Various systems and methods for graphicallyaccentuating content listings are discussed in, for example, Yates, U.S.Patent Application Publication No. 2010/0153885, filed Nov. 12, 2009,which is hereby incorporated by reference herein in its entirety.

Users may access content, the media guidance application (and itsdisplay screens described above and below from one or more of their userequipment devices (for example, heads up display 102 or head-mounteddisplay 301). FIG. 8 shows a generalized embodiment of illustrative userequipment device 600. More specific implementations of user equipmentdevices are discussed below in connection with FIG. 9. User equipmentdevice 600 may receive content and data via input/output (hereinafter“I/O”) path 602. I/O path 602 may provide content (e.g., broadcastprogramming, on-demand programming, Internet content, content availableover a local area network (LAN) or wide area network (WAN), and/or othercontent) and data to control circuitry 604, which includes processingcircuitry 606 and storage 608. Control circuitry 604 may be used to sendand receive commands, requests, and other suitable data using I/O path602. I/O path 602 may connect control circuitry 604 (and specificallyprocessing circuitry 606) to one or more communications paths (describedbelow). I/O functions may be provided by one or more of thesecommunications paths, but are shown as a single path in FIG. 8 to avoidovercomplicating the drawing.

Control circuitry 604 may be based on any suitable processing circuitrysuch as processing circuitry 606. As referred to herein, processingcircuitry should be understood to mean circuitry based on one or moremicroprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry may be distributed across multiple separateprocessors or processing units, for example, multiple of the same typeof processing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). In some embodiments, control circuitry 604 executesinstructions for an application stored in memory (i.e., storage 608).Specifically, control circuitry 604 may be instructed by the applicationto perform the functions discussed above and below. For example, theapplication may provide instructions to control circuitry 604 togenerate the displays. In some implementations, any action performed bycontrol circuitry 604 may be based on instructions received from theapplication.

In client-server based embodiments, control circuitry 604 may includecommunications circuitry suitable for communicating with a guidanceapplication server or other networks or servers. The instructions forcarrying out the above mentioned functionality may be stored on theguidance application server. Communications circuitry may include acable modem, an integrated services digital network (ISDN) modem, adigital subscriber line (DSL) modem, a telephone modem, Ethernet card,or a wireless modem for communications with other equipment, or anyother suitable communications circuitry. Such communications may involvethe Internet or any other suitable communications networks or paths(which is described in more detail in connection with FIG. 9). Inaddition, communications circuitry may include circuitry that enablespeer-to-peer communication of user equipment devices, or communicationof user equipment devices in locations remote from each other (describedin more detail below).

Memory may be an electronic storage device provided as storage 608 thatis part of control circuitry 604. As referred to herein, the phrase“electronic storage device” or “storage device” should be understood tomean any device for storing electronic data, computer software, orfirmware, such as random-access memory, read-only memory, hard drives,optical drives, digital video disc (DVD) recorders, compact disc (CD)recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders,digital video recorders (DVR, sometimes called a personal videorecorder, or PVR), solid state devices, quantum storage devices, gamingconsoles, gaming media, or any other suitable fixed or removable storagedevices, and/or any combination of the same. Storage 608 may be used tostore various types of content described herein as well as mediaguidance data described above. Nonvolatile memory may also be used(e.g., to launch a boot-up routine and other instructions). Cloud-basedstorage, described in relation to FIG. 9, may be used to supplementstorage 608 or instead of storage 608.

Control circuitry 604 may include video generating circuitry and tuningcircuitry, such as one or more analog tuners, one or more MPEG-2decoders or other digital decoding circuitry, high-definition tuners, orany other suitable tuning or video circuits or combinations of suchcircuits. Encoding circuitry (e.g., for converting over-the-air, analog,or digital signals to MPEG signals for storage) may also be provided.Control circuitry 604 may also include scaler circuitry for upconvertingand downconverting content into the preferred output format of the userequipment 600. Circuitry 604 may also include digital-to-analogconverter circuitry and analog-to-digital converter circuitry forconverting between digital and analog signals. The tuning and encodingcircuitry may be used by the user equipment device to receive and todisplay, to play, or to record content. The tuning and encodingcircuitry may also be used to receive guidance data. The circuitrydescribed herein, including for example, the tuning, video generating,encoding, decoding, encrypting, decrypting, scaler, and analog/digitalcircuitry, may be implemented using software running on one or moregeneral purpose or specialized processors. Multiple tuners may beprovided to handle simultaneous tuning functions (e.g., watch and recordfunctions, picture-in-picture (PIP) functions, multiple-tuner recording,etc.). If storage 608 is provided as a separate device from userequipment 600, the tuning and encoding circuitry (including multipletuners) may be associated with storage 608.

A user may send instructions to control circuitry 604 using user inputinterface 610. User input interface 610 may be any suitable userinterface, such as a remote control, mouse, trackball, keypad, keyboard,touch screen, touchpad, stylus input, joystick, voice recognitioninterface, or other user input interfaces. Display 612 may be providedas a stand-alone device or integrated with other elements of userequipment device 600. For example, display 612 may be a touchscreen ortouch-sensitive display. In such circumstances, user input interface 610may be integrated with or combined with display 612. Display 612 may beone or more of a monitor, a television, a liquid crystal display (LCD)for a mobile device, amorphous silicon display, low temperature polysilicon display, electronic ink display, electrophoretic display, activematrix display, electro-wetting display, electrofluidic display, cathoderay tube display, light-emitting diode display, electroluminescentdisplay, plasma display panel, high-performance addressing display,thin-film transistor display, organic light-emitting diode display,surface-conduction electron-emitter display (SED), laser television,carbon nanotubes, quantum dot display, interferometric modulatordisplay, or any other suitable equipment for displaying visual images.In some embodiments, display 612 may be HDTV-capable. In someembodiments, display 612 may be a 3D display, and the interactive mediaguidance application and any suitable content may be displayed in 3D. Avideo card or graphics card may generate the output to the display 612.The video card may offer various functions such as accelerated renderingof 3D scenes and 2D graphics, MPEG-2/MPEG-4 decoding, TV output, or theability to connect multiple monitors. The video card may be anyprocessing circuitry described above in relation to control circuitry604. The video card may be integrated with the control circuitry 604.Speakers 614 may be provided as integrated with other elements of userequipment device 600 or may be stand-alone units. The audio component ofvideos and other content displayed on display 612 may be played throughspeakers 614. In some embodiments, the audio may be distributed to areceiver (not shown), which processes and outputs the audio via speakers614.

The guidance application may be implemented using any suitablearchitecture. For example, it may be a stand-alone applicationwholly-implemented on user equipment device 600. In such an approach,instructions of the application are stored locally (e.g., in storage608), and data for use by the application is downloaded on a periodicbasis (e.g., from an out-of-band feed, from an Internet resource, orusing another suitable approach). Control circuitry 604 may retrieveinstructions of the application from storage 608 and process theinstructions to generate any of the displays discussed herein. Based onthe processed instructions, control circuitry 604 may determine whataction to perform when input is received from input interface 610. Forexample, movement of a cursor on a display up/down may be indicated bythe processed instructions when input interface 610 indicates that anup/down button was selected.

In some embodiments, the media guidance application is a client-serverbased application. Data for use by a thick or thin client implemented onuser equipment device 600 is retrieved on-demand by issuing requests toa server remote to the user equipment device 600. In one example of aclient-server based guidance application, control circuitry 604 runs aweb browser that interprets web pages provided by a remote server. Forexample, the remote server may store the instructions for theapplication in a storage device. The remote server may process thestored instructions using circuitry (e.g., control circuitry 604) andgenerate the displays discussed above and below. The client device mayreceive the displays generated by the remote server and may display thecontent of the displays locally on equipment device 600. This way, theprocessing of the instructions is performed remotely by the server whilethe resulting displays are provided locally on equipment device 600.Equipment device 600 may receive inputs from the user via inputinterface 610 and transmit those inputs to the remote server forprocessing and generating the corresponding displays. For example,equipment device 600 may transmit a communication to the remote serverindicating that an up/down button was selected via input interface 610.The remote server may process instructions in accordance with that inputand generate a display of the application corresponding to the input(e.g., a display that moves a cursor up/down). The generated display isthen transmitted to equipment device 600 for presentation to the user.

In some embodiments, the media guidance application is downloaded andinterpreted or otherwise run by an interpreter or virtual machine (runby control circuitry 604). In some embodiments, the guidance applicationmay be encoded in the ETV Binary Interchange Format (EBIF), received bycontrol circuitry 604 as part of a suitable feed, and interpreted by auser agent running on control circuitry 604. For example, the guidanceapplication may be an EBIF application. In some embodiments, theguidance application may be defined by a series of JAVA-based files thatare received and run by a local virtual machine or other suitablemiddleware executed by control circuitry 604. In some of suchembodiments (e.g., those employing MPEG-2 or other digital mediaencoding schemes), the guidance application may be, for example, encodedand transmitted in an MPEG-2 object carousel with the MPEG audio andvideo packets of a program.

Control circuitry 604 includes detection module 616. Detection module616 will be discussed in further detail in relation to FIGS. 10-12.

User equipment device 600 of FIG. 8 can be implemented in system 700 ofFIG. 9 as user television equipment 702, user computer equipment 704,wireless user communications device 706, a heads up display (forexample, head-mounted 301) or any other type of user equipment suitablefor accessing content, such as a non-portable gaming machine. Forsimplicity, these devices may be referred to herein collectively as userequipment or user equipment devices, and may be substantially similar touser equipment devices described above. User equipment devices, on whicha media guidance application may be implemented, may function as astandalone device or may be part of a network of devices. Variousnetwork configurations of devices may be implemented and are discussedin more detail below.

A user equipment device utilizing at least some of the system featuresdescribed above in connection with FIG. 8 may not be classified solelyas user television equipment 702, user computer equipment 704, or awireless user communications device 706. For example, user televisionequipment 702 may, like some user computer equipment 704, beInternet-enabled allowing for access to Internet content, while usercomputer equipment 704 may, like some user television equipment 702,include a tuner allowing for access to television programming. The mediaguidance application may have the same layout on various different typesof user equipment or may be tailored to the display capabilities of theuser equipment. For example, on user computer equipment 704, theguidance application may be provided as a web site accessed by a webbrowser. In another example, the guidance application may be scaled downfor wireless user communications devices 706.

In system 700, there is typically more than one of each type of userequipment device but only one of each is shown in FIG. 9 to avoidovercomplicating the drawing. In addition, each user may utilize morethan one type of user equipment device and also more than one of eachtype of user equipment device.

In some embodiments, a user equipment device (e.g., user televisionequipment 702, user computer equipment 704, wireless user communicationsdevice 706) may be referred to as a “second screen device.” For example,a second screen device may supplement content presented on a first userequipment device. The content presented on the second screen device maybe any suitable content that supplements the content presented on thefirst device. In some embodiments, the second screen device provides aninterface for adjusting settings and display preferences of the firstdevice. In some embodiments, the second screen device is configured forinteracting with other second screen devices or for interacting with asocial network. The second screen device can be located in the same roomas the first device, a different room from the first device but in thesame house or building, or in a different building from the firstdevice.

The user may also set various settings to maintain consistent mediaguidance application settings across in-home devices and remote devices.Settings include those described herein, as well as channel and programfavorites, programming preferences that the guidance applicationutilizes to make programming recommendations, display preferences, andother desirable guidance settings. For example, if a user sets a channelas a favorite on, for example, the web site www.allrovi.com on theirpersonal computer at their office, the same channel would appear as afavorite on the user's in-home devices (e.g., user television equipmentand user computer equipment) as well as the user's mobile devices, ifdesired. Therefore, changes made on one user equipment device can changethe guidance experience on another user equipment device, regardless ofwhether they are the same or a different type of user equipment device.In addition, the changes made may be based on settings input by a user,as well as user activity monitored by the guidance application.

The user equipment devices may be coupled to communications network 714.Namely, user television equipment 702, user computer equipment 704, andwireless user communications device 706 are coupled to communicationsnetwork 714 via communications paths 708, 710, and 712, respectively.Communications network 714 may be one or more networks including theInternet, a mobile phone network, mobile voice or data network (e.g., a4G or LTE network), cable network, public switched telephone network, orother types of communications network or combinations of communicationsnetworks. Paths 708, 710, and 712 may separately or together include oneor more communications paths, such as, a satellite path, a fiber-opticpath, a cable path, a path that supports Internet communications (e.g.,IPTV), free-space connections (e.g., for broadcast or other wirelesssignals), or any other suitable wired or wireless communications path orcombination of such paths. Path 712 is drawn with dotted lines toindicate that in the exemplary embodiment shown in FIG. 9 it is awireless path and paths 708 and 710 are drawn as solid lines to indicatethey are wired paths (although these paths may be wireless paths, ifdesired). Communications with the user equipment devices may be providedby one or more of these communications paths, but are shown as a singlepath in FIG. 9 to avoid overcomplicating the drawing.

Although communications paths are not drawn between user equipmentdevices, these devices may communicate directly with each other viacommunication paths, such as those described above in connection withpaths 708, 710, and 712, as well as other short-range point-to-pointcommunication paths, such as USB cables, IEEE 1394 cables, wirelesspaths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.), or othershort-range communication via wired or wireless paths. BLUETOOTH is acertification mark owned by Bluetooth SIG, INC. The user equipmentdevices may also communicate with each other directly through anindirect path via communications network 714.

System 700 includes content source 716 and media guidance data source718 coupled to communications network 714 via communication paths 720and 722, respectively. Paths 720 and 722 may include any of thecommunication paths described above in connection with paths 708, 710,and 712. Communications with the content source 716 and media guidancedata source 718 may be exchanged over one or more communications paths,but are shown as a single path in FIG. 9 to avoid overcomplicating thedrawing. In addition, there may be more than one of each of contentsource 716 and media guidance data source 718, but only one of each isshown in FIG. 9 to avoid overcomplicating the drawing. (The differenttypes of each of these sources are discussed below.) If desired, contentsource 716 and media guidance data source 718 may be integrated as onesource device. Although communications between sources 716 and 718 withuser equipment devices 702, 704, and 706 are shown as throughcommunications network 714, in some embodiments, sources 716 and 718 maycommunicate directly with user equipment devices 702, 704, and 706 viacommunication paths (not shown) such as those described above inconnection with paths 708, 710, and 712.

Content source 716 may include one or more types of content distributionequipment including a television distribution facility, cable systemheadend, satellite distribution facility, programming sources (e.g.,television broadcasters, such as NBC, ABC, HBO, etc.), intermediatedistribution facilities and/or servers, Internet providers, on-demandmedia servers, and other content providers. NBC is a trademark owned bythe National Broadcasting Company, Inc., ABC is a trademark owned by theAmerican Broadcasting Company, Inc., and HBO is a trademark owned by theHome Box Office, Inc. Content source 716 may be the originator ofcontent (e.g., a television broadcaster, a Webcast provider, etc.) ormay not be the originator of content (e.g., an on-demand contentprovider, an Internet provider of content of broadcast programs fordownloading, etc.). Content source 716 may include cable sources,satellite providers, on-demand providers, Internet providers,over-the-top content providers, or other providers of content. Contentsource 716 may also include a remote media server used to storedifferent types of content (including video content selected by a user),in a location remote from any of the user equipment devices. Systems andmethods for remote storage of content, and providing remotely storedcontent to user equipment are discussed in greater detail in connectionwith Ellis et al., U.S. Pat. No. 7,761,892, issued Jul. 20, 2010, whichis hereby incorporated by reference herein in its entirety.

Media guidance data source 718 may provide media guidance data, such asthe media guidance data described above. Media guidance data may beprovided to the user equipment devices using any suitable approach. Insome embodiments, the guidance application may be a stand-aloneinteractive television program guide that receives program guide datavia a data feed (e.g., a continuous feed or trickle feed). Programschedule data and other guidance data may be provided to the userequipment on a television channel sideband, using an in-band digitalsignal, using an out-of-band digital signal, or by any other suitabledata transmission technique. Program schedule data and other mediaguidance data may be provided to user equipment on multiple analog ordigital television channels.

In some embodiments, guidance data from media guidance data source 718may be provided to users' equipment using a client-server approach. Forexample, a user equipment device may pull media guidance data from aserver, or a server may push media guidance data to a user equipmentdevice. In some embodiments, a guidance application client residing onthe user's equipment may initiate sessions with source 718 to obtainguidance data when needed, e.g., when the guidance data is out of dateor when the user equipment device receives a request from the user toreceive data. Media guidance may be provided to the user equipment withany suitable frequency (e.g., continuously, daily, a user-specifiedperiod of time, a system-specified period of time, in response to arequest from user equipment, etc.). Media guidance data source 718 mayprovide user equipment devices 702, 704, and 706 the media guidanceapplication itself or software updates for the media guidanceapplication.

In some embodiments, the media guidance data may include viewer data.For example, the viewer data may include current and/or historical useractivity information (e.g., what content the user typically watches,what times of day the user watches content, whether the user interactswith a social network, at what times the user interacts with a socialnetwork to post information, what types of content the user typicallywatches (e.g., pay TV or free TV), mood, brain activity information,etc.). The media guidance data may also include subscription data. Forexample, the subscription data may identify to which sources or servicesa given user subscribes and/or to which sources or services the givenuser has previously subscribed but later terminated access (e.g.,whether the user subscribes to premium channels, whether the user hasadded a premium level of services, whether the user has increasedInternet speed). In some embodiments, the viewer data and/or thesubscription data may identify patterns of a given user for a period ofmore than one year. The media guidance data may include a model (e.g., asurvivor model) used for generating a score that indicates a likelihooda given user will terminate access to a service/source. For example, themedia guidance application may process the viewer data with thesubscription data using the model to generate a value or score thatindicates a likelihood of whether the given user will terminate accessto a particular service or source. In particular, a higher score mayindicate a higher level of confidence that the user will terminateaccess to a particular service or source. Based on the score, the mediaguidance application may generate promotions and advertisements thatentice the user to keep the particular service or source indicated bythe score as one to which the user will likely terminate access.

Media guidance applications may be, for example, stand-aloneapplications implemented on user equipment devices. For example, themedia guidance application may be implemented as software or a set ofexecutable instructions which may be stored in storage 608, and executedby control circuitry 604 of a user equipment device 600. In someembodiments, media guidance applications may be client-serverapplications where only a client application resides on the userequipment device, and server application resides on a remote server. Forexample, media guidance applications may be implemented partially as aclient application on control circuitry 604 of user equipment device 600and partially on a remote server as a server application (e.g., mediaguidance data source 718) running on control circuitry of the remoteserver. When executed by control circuitry of the remote server (such asmedia guidance data source 718), the media guidance application mayinstruct the control circuitry to generate the guidance applicationdisplays and transmit the generated displays to the user equipmentdevices. The server application may instruct the control circuitry ofthe media guidance data source 718 to transmit data for storage on theuser equipment. The client application may instruct control circuitry ofthe receiving user equipment to generate the guidance applicationdisplays.

Content and/or media guidance data delivered to user equipment devices702, 704, and 706 may be over-the-top (OTT) content. OTT contentdelivery allows Internet-enabled user devices, including any userequipment device described above, to receive content that is transferredover the Internet, including any content described above, in addition tocontent received over cable or satellite connections. OTT content isdelivered via an Internet connection provided by an Internet serviceprovider (ISP), but a third party distributes the content. The ISP maynot be responsible for the viewing abilities, copyrights, orredistribution of the content, and may only transfer IP packets providedby the OTT content provider. Examples of OTT content providers includeYOUTUBE, NETFLIX, and HULU, which provide audio and video via IPpackets. Youtube is a trademark owned by Google Inc., Netflix is atrademark owned by Netflix Inc., and Hulu is a trademark owned by Hulu,LLC. OTT content providers may additionally or alternatively providemedia guidance data described above. In addition to content and/or mediaguidance data, providers of OTT content can distribute media guidanceapplications (e.g., web-based applications or cloud-based applications),or the content can be displayed by media guidance applications stored onthe user equipment device.

Media guidance system 700 is intended to illustrate a number ofapproaches, or network configurations, by which user equipment devicesand sources of content and guidance data may communicate with each otherfor the purpose of accessing content and providing media guidance. Theembodiments described herein may be applied in any one or a subset ofthese approaches, or in a system employing other approaches fordelivering content and providing media guidance. The following fourapproaches provide specific illustrations of the generalized example ofFIG. 9.

In one approach, user equipment devices may communicate with each otherwithin a home network. User equipment devices can communicate with eachother directly via short-range point-to-point communication schemesdescribed above, via indirect paths through a hub or other similardevice provided on a home network, or via communications network 714.Each of the multiple individuals in a single home may operate differentuser equipment devices on the home network. As a result, it may bedesirable for various media guidance information or settings to becommunicated between the different user equipment devices. For example,it may be desirable for users to maintain consistent media guidanceapplication settings on different user equipment devices within a homenetwork, as described in greater detail in Ellis et al., U.S. PatentPublication No. 2005/0251827, filed Jul. 11, 2005. Different types ofuser equipment devices in a home network may also communicate with eachother to transmit content. For example, a user may transmit content fromuser computer equipment to a portable video player or portable musicplayer.

In a second approach, users may have multiple types of user equipment bywhich they access content and obtain media guidance. For example, someusers may have home networks that are accessed by in-home and mobiledevices. Users may control in-home devices via a media guidanceapplication implemented on a remote device. For example, users mayaccess an online media guidance application on a website via a personalcomputer at their office, or a mobile device such as a PDA orweb-enabled mobile telephone. The user may set various settings (e.g.,recordings, reminders, or other settings) on the online guidanceapplication to control the user's in-home equipment. The online guidemay control the user's equipment directly, or by communicating with amedia guidance application on the user's in-home equipment. Varioussystems and methods for user equipment devices communicating, where theuser equipment devices are in locations remote from each other, isdiscussed in, for example, Ellis et al., U.S. Pat. No. 8,046,801, issuedOct. 25, 2011, which is hereby incorporated by reference herein in itsentirety.

In a third approach, users of user equipment devices inside and outsidea home can use their media guidance application to communicate directlywith content source 716 to access content. Specifically, within a home,users of user television equipment 702 and user computer equipment 704may access the media guidance application to navigate among and locatedesirable content. Users may also access the media guidance applicationoutside of the home using wireless user communications devices 706 tonavigate among and locate desirable content.

In a fourth approach, user equipment devices may operate in a cloudcomputing environment to access cloud services. In a cloud computingenvironment, various types of computing services for content sharing,storage or distribution (e.g., video sharing sites or social networkingsites) are provided by a collection of network-accessible computing andstorage resources, referred to as “the cloud.” For example, the cloudcan include a collection of server computing devices, which may belocated centrally or at distributed locations, that provide cloud-basedservices to various types of users and devices connected via a networksuch as the Internet via communications network 714. These cloudresources may include one or more content sources 716 and one or moremedia guidance data sources 718. In addition or in the alternative, theremote computing sites may include other user equipment devices, such asuser television equipment 702, user computer equipment 704, and wirelessuser communications device 706. For example, the other user equipmentdevices may provide access to a stored copy of a video or a streamedvideo. In such embodiments, user equipment devices may operate in apeer-to-peer manner without communicating with a central server.

The cloud provides access to services, such as content storage, contentsharing, or social networking services, among other examples, as well asaccess to any content described above, for user equipment devices.Services can be provided in the cloud through cloud computing serviceproviders, or through other providers of online services. For example,the cloud-based services can include a content storage service, acontent sharing site, a social networking site, or other services viawhich user-sourced content is distributed for viewing by others onconnected devices. These cloud-based services may allow a user equipmentdevice to store content to the cloud and to receive content from thecloud rather than storing content locally and accessing locally-storedcontent.

A user may use various content capture devices, such as camcorders,digital cameras with video mode, audio recorders, mobile phones, andhandheld computing devices, to record content. The user can uploadcontent to a content storage service on the cloud either directly, forexample, from user computer equipment 704 or wireless usercommunications device 706 having content capture feature. Alternatively,the user can first transfer the content to a user equipment device, suchas user computer equipment 704. The user equipment device storing thecontent uploads the content to the cloud using a data transmissionservice on communications network 714. In some embodiments, the userequipment device itself is a cloud resource, and other user equipmentdevices can access the content directly from the user equipment deviceon which the user stored the content.

Cloud resources may be accessed by a user equipment device using, forexample, a web browser, a media guidance application, a desktopapplication, a mobile application, and/or any combination of accessapplications of the same. The user equipment device may be a cloudclient that relies on cloud computing for application delivery, or theuser equipment device may have some functionality without access tocloud resources. For example, some applications running on the userequipment device may be cloud applications, i.e., applications deliveredas a service over the Internet, while other applications may be storedand run on the user equipment device. In some embodiments, a user devicemay receive content from multiple cloud resources simultaneously. Forexample, a user device can stream audio from one cloud resource whiledownloading content from a second cloud resource. Or a user device candownload content from multiple cloud resources for more efficientdownloading. In some embodiments, user equipment devices can use cloudresources for processing operations such as the processing operationsperformed by processing circuitry described in relation to FIG. 8.

As referred herein, the term “in response to” refers to initiated as aresult of. For example, a first action being performed in response to asecond action may include interstitial steps between the first actionand the second action. As referred herein, the term “directly inresponse to” refers to caused by. For example, a first action beingperformed directly in response to a second action may not includeinterstitial steps between the first action and the second action.

FIG. 10 is an illustrative example of one component of detection module616, which may be accessed in accordance with some embodiments of thedisclosure. FIG. 10 shows gaze point detection component 800, which maybe used to identify the center of gaze position of a user 808.

Gaze point detection component 800 includes processor 802, light source804, and optical sensor 806. Light source 804 transmits light thatreaches at least one eye of user 808, and optical sensor 806 is directedat user 808 to sense reflected light. Optical sensor 806 transmitscollected data to processor 802, and based on the data received fromoptical sensor 806, processor 802 determines the gaze point of the user808.

Processor 802 may be integrated with one or more light sources 804 andone or more optical sensors 806 in a single device. Additionally oralternatively, one light source 804 may transmit light to both eyes ofuser 808. Additionally or alternatively, one light source 804 maytransmit light to one eye of user 808 and another light source 804 maytransmit light to the other eye of user 808. Additionally oralternatively, one or more light sources 804 and one or more opticalsensors 806 may be housed separately from processor 802 and in wirelessor wired communication with processor 802. One or more of processors802, light sources 804, and optical sensors 806 may be integrated intouser equipment device 600.

Processor 802 may be similar to processing circuitry 606 describedabove. In some embodiments, processor 802 may be processing circuitry606, with processing circuitry 606 in communication with light source804 and optical sensor 806. In other embodiments, processor 802 may beseparate from but optionally in communication with processing circuitry606.

Light source 804 transmits light to one or both eyes of one or moreusers. Light source 804 may emit, for example, infrared (IR) light, nearinfrared light, or visible light. The light emitted by light source 804may be collimated or non-collimated. The light is reflected in a user'seye, forming, for example, the reflection from the outer surface of thecornea (i.e. a first Purkinje image), the reflection from the innersurface of the cornea (i.e. a second Purkinje image), the reflectionfrom the outer (anterior) surface of the lens (i.e. a third Purkinjeimage), and/or the reflection from the inner (posterior) surface of thelens (i.e. a fourth Purkinje image).

Optical sensor 806 collects visual information, such as an image orseries of images, of one or both of one or more users' eyes. Opticalsensor 806 transmits the collected image(s) to processor 802, whichprocesses the received image(s) to detect location of a glint (i.e.corneal reflection) and/or other reflection in one or both eyes of oneor more users. Processor 802 may also determine the location of thepupil of one or both eyes of one or more users. For each eye, processor802 may compare the location of the pupil to the location of the glintand/or other reflection to determine the gaze point of each eye.Processor 802 may also store or obtain information describing thelocation of one or more light sources 804 and/or the location of one ormore optical sensors 806 relative to display 612. Using thisinformation, processor 802 may determine a gaze point of each eye ofuser 808 relative to display 612. Processor 802 may calculate themidpoint between the gaze point of each eye of user 808 to determine thecenter of gaze position of user 808. Processor 802 may compare thecurrent center of gaze position of user 808 to a previously-stored (forexample, in storage 608) center of gaze position of user 808 todetermine that the center of gaze of user 808 has moved.

In some embodiments, gaze point detection component 800 performs best ifthe position of a user's head is fixed or relatively stable. In otherembodiments, gaze point detection component 800 is configured to accountfor a user's head movement, which allows the user a more natural viewingexperience than if the user's head were fixed in a particular position.

In some embodiments accounting for a user's head movement, gaze pointdetection component 800 includes two or more optical sensors 806. Forexample, two cameras may be arranged to form a stereo vision system forobtaining a 3D position of the user's eye or eyes; this allows processor802 to compensate for head movement when determining the user's gazepoint. The two or more optical sensors 806 may be part of a single unitor may be separate units. For example, user equipment device 600 mayinclude two cameras used as optical sensors 806, or gaze point detectioncomponent 800 in communication with user equipment device 600 mayinclude two optical sensors 806. In other embodiments, each of userequipment device 600 and gaze point detection component 800 may includean optical sensor, and processor 802 receives image data from theoptical sensor of user equipment device 600 and the optical sensor ofgaze point detection component 800. Processor 802 may receive dataidentifying the location of optical sensor 806 relative to the display612 and/or relative to each other and use this information whendetermining the gaze point.

In other embodiments accounting for a user's head movement, gaze pointdetection component 800 includes two or more light sources forgenerating multiple glints. For example, two light sources 804 maycreate glints at different locations of an eye; having information onthe two glints allows the processor to determine a 3D position of theuser's eye or eyes, allowing processor 802 to compensate for headmovement. Processor 802 may also receive data identifying the locationof light sources 804 relative to the display 612 and/or relative to eachother and use this information when determining the gaze point.

In some embodiments, gaze point detection component 800 is configured toaccount for visual impairments of users. For example, if user 808 isblind in one eye, gaze point detection component 800 may not transmitlight to that eye and not base its determination of center of gaze onthat eye. For example, if user 808 is blind in the left eye, the centerof gaze may be determined to be the gaze point of the right eye. Asanother example, if user 808 has a lazy eye, gaze point detectioncomponent 800 may not transmit light to that eye and not base itsdetermination of center of gaze on that eye. As another example, if user808 has a lazy left eye and a movement of the left eye of the user 808is detected, gaze point detection component 800 may be configured toignore that movement.

In some embodiments, other types of gaze point detection components thatdo not utilize a light source may be used. For example, optical sensor806 and processor 802 may track other features of a user's eye, such asthe retinal blood vessels or other features inside or on the surface ofthe user's eye, and follow these features as the eye rotates. Any otherequipment or method for determining one or more users' gaze point(s) notdiscussed above may be used in addition to or instead of theabove-described embodiments of gaze point detection component 800.

It should be noted that gaze point detection component 800 is but onetype of component that may be incorporated into or accessible bydetection module 616. Other types of components, which may generateother types of data (e.g., video, audio, textual, etc.) are fully withinthe bounds of this disclosure.

FIG. 11 is another illustrative example of one component of detectionmodule 616, which may be accessed in accordance with some embodiments ofthe disclosure. FIG. 11 shows head movement detection component 900,which may be used to detect movement of the head of user 908. Headmovement detection component 900 includes processor 902 andaccelerometer 904. Accelerometer 904 is physically coupled to the user'shead. For example, if user equipment device 600 is a head-mounteddisplay physically coupled to the head of the user 908 (for example,head-mounted display 301 of FIG. 5), accelerometer 904 may be includedin the head-mounted display. Accelerometer 904 may be capable ofdetecting its own acceleration in one dimension, two dimensions, orthree dimensions. Accelerometer 904 may be a capacitive accelerometer, apiezoelectric accelerometer, a micro-electromechanical (MEMS)accelerometer, or any other type of accelerometer.

Processor 902 may be integrated with one or more accelerometers 904 in asingle device. Additionally or alternatively, one or more accelerometers904 may be housed separately from processor 902 and in wireless or wiredcommunication with processor 902. One or more of processors 902 andaccelerometers 904 may be integrated into user equipment device 600.

Processor 902 may be similar to processing circuitry 606 describedabove. In some embodiments, processor 902 may be processing circuitry606, with processing circuitry 606 in communication with accelerometer904. In other embodiments, processor 902 may be separate from butoptionally in communication with processing circuitry 906.

If the user's head moves or rotates in a certain direction,accelerometer 904 will detect the acceleration transmit an output toprocessor 902 that the user's head is accelerating in that direction fora certain amount of time. Additionally, accelerometer 904 may transmitan output to processor 902 that the user's head is accelerating with acertain acceleration magnitude.

Processor 902 may store or obtain information describing the location ofone or more accelerometers 904 relative to the display 612. Using thisinformation, as well as information about the direction, duration, andmagnitude of the acceleration of the user's head as received fromaccelerometer 904, processor 902 may determine a new position of thehead of user 908 relative to display 612. Processor 902 may store orobtain information describing the location of one or more accelerometers904 relative to the eyes of user 908. Assuming that user 908 is lookingstraight ahead, processor 902 may use this information as well asinformation about the new position of the head of user 908 to detectthat the center of gaze of user 908 has moved and to determine the newcenter of gaze position of user 908.

Accelerometer 904 may be configured to detect a footstep of a user bydetecting movements of the user's head upwards, forwards, and downwardsin succession. In some embodiments, accelerometer 904 is configured touse calibration data to detect footsteps of a user. In such embodiments,a user may take a footstep and accelerometer 904 may save parameterssuch as acceleration direction, magnitude, and timing during thefootstep. Future movements substantially matching these parameters maybe identified by accelerometer 904 as a footstep.

Accelerometer 904 may output to processor 902 that the user took afootstep in response to detecting the footstep. Processor 902 maydetermine, based on the output of the accelerometer 904, that the userperformed a full body movement. In some embodiments, in response todetection of a first footstep, control circuitry 604 generates fordisplay additional content (for example, additional content 222) in aportion of a virtual reality environment (for example, virtual realityenvironment 104) corresponding to a foreground area of a user's visualfield, as described in relation to FIGS. 3-4. In some embodiments, inresponse to detection of a second footstep, control circuitry 604enlarges the size of the additional content. In some embodiments, inresponse to detection of a second footstep, control circuitry 604performs at least one of decreasing the opacity of the main content andincreasing the opacity of the additional content. These features may bebeneficial if, for example, the user wishes to focus more on theadditional content and less on the main content while performing thefull body movement.

If accelerometer 904 detects no substantial acceleration of the head ofuser 908, accelerometer 904 may output to processor 902 that the user issubstantially stationary. In some implementations, in response todetection that the user is substantially stationary, control circuitry604 generates for display the additional content in a portion of thedisplay that is not in the foreground area of a user's visual field.

It should be noted that head movement detection component 900 is but onetype of component that may be incorporated into or accessible bydetection module 616.

Other types of components, which may generate other types of data (e.g.,video, audio, textual, etc.) are fully within the bounds of thisdisclosure.

FIG. 12 is another illustrative example of one component of detectionmodule 616, which may be accessed in accordance with some embodiments ofthe disclosure. FIG. 12 shows movement detection component 1000, whichmay be used to detect movement of a user 1008 by detecting a change in avideo of the user's physical surroundings. Movement detection component1000 includes processor 1002 and camera 1004. Camera 1004 is physicallycoupled to user 1008. For example, camera 1004 may be coupled to thehead of the user 1008. Camera 1004 may be a camera of a mobile phoneincluded in a head-mounted display (for example, camera 317).

Camera 1004 is configured to capture images and/or videos of the user'sphysical surroundings and compare the captured images/videos to detectmovement of the user. For example, as illustrated in FIG. 12, camera1004 captures frame 1006 in a video of the user's physical surroundingsand then captures frame 1008 in the video of the user's physicalsurroundings. Processor 1002 is configured to compare frame 1006 andframe 1008 and detect a change in the video of the user's physicalsurroundings. In FIG. 12, processor 1002 will determine that the user1008 has moved to the right. In response to this determination,processor 1002 may determine that the center of gaze of the user haschanged. Additionally or alternatively, in response to thisdetermination, processor 1002 may determine that the user is performinga full body movement.

In some embodiments, occurrences in the user's physical surroundings(e.g., another person walking into the room in which the user islocated, another person knocking on the door of the house in which theuser is located, something dangerous occurring in the vicinity of theuser, etc.) may be detected. For example, movement detection component1000 may be configured to detect occurrences in the user's physicalsurroundings by capturing multiple frames of video of the user'sphysical surroundings and comparing the multiple frames to detectchanges. In response to detecting an occurrence in the user's physicalsurroundings, user input interface 610 may present information to theuser regarding the occurrence. For example, user input interface 610 mayoutput a sound through speakers 614 indicating the occurrence or userinput interface 610 may output text on display 612 indicating theoccurrence. The indication of the occurrence may be a statementinforming the user of the occurrence, for example, text or sound saying,“Someone has walked into the room.” In some embodiments, user inputinterface 610 is configured to present the user with an option (forexample, through speakers 614 or display 612) to react to theoccurrence. For example, if the occurrence is a knock at a door, userinput interface 610 may present the user with an option to automaticallyopen the door. The user may respond to the option through voice, typingon a keyboard, or any other means of input. In some embodiments, controlcircuitry 604 is configured to present the user with additionalinformation (for example, additional information 122 or 222) to assistthe user in reacting to the occurrence. For example, control circuitry604 may generate for display additional information including a video ofthe occurrence. For example, if the occurrence is a knock at a door, theadditional information may be a video of the person at the door. In someembodiments, the additional information is generated for display in aportion of the virtual reality environment (e.g., virtual realityenvironments 104 or 204) corresponding to a foreground area of theuser's visual field so the user can easily see the additionalinformation. In some embodiments, the additional information replacesthe additional information generated for display before the occurrence.

In some embodiments, occurrences in the user's physical surroundings aredetected by one or more sensors (for example, sound sensors, temperaturesensors, etc.). For example, a temperature sensor may detect a personwalking into the room in which the user is located by detectingincreased heat in the room. The one or more sensors may be integratedinto user equipment device 600 or may be external to user equipmentdevice 600, in which case the one or more sensors 600 are configured totransmit sensed information to user equipment device 600, for example,over a wireless network.

In some embodiments, control circuitry 604 is configured to determine avisual field (for example, second visual field 126) in response to amovement of the center of gaze of a user. In some embodiments, controlcircuitry 604 determines the new center of gaze of the user (forexample, using gaze point detection component 800 or head movementdetection component 900). The visual field of the user encompasses areaswithin a first number of degrees to the right and left of the new centerof gaze, a second number of degrees above the center of gaze, and athird number of degrees below the center of gaze.

In some embodiments, the first number of degrees is equal to or greaterthan 95 degrees, for example, 95 degrees, 100 degrees, 105 degrees, 110degrees, 115 degrees, 120 degrees, >120 degrees, or any suitable numberof degrees. In some embodiments, the first number of degrees is lessthan 95 degrees, for example, 90 degrees, 85 degrees, 80 degrees, 75degrees, 70 degrees, <70 degrees, or any suitable number of degrees. Insome embodiments, the second number of degrees is equal to or greaterthan 60 degrees, for example, 65 degrees, 70 degrees, 75 degrees, 80degrees, 85 degrees, >85 degrees, or any suitable number of degrees. Insome embodiments, the second number of degrees is less than 60 degrees,for example, 55 degrees, 50 degrees, 45 degrees, 40 degrees, <40degrees, or any suitable number of degrees. In some embodiments, thethird number of degrees is equal to or greater than 70 degrees, forexample, 75 degrees, 80 degrees, 85 degrees, 90 degrees, >90 degrees, orany suitable number of degrees. In some embodiments, the third number ofdegrees is less than 70 degrees, for example, 65 degrees, 60 degrees, 55degrees, <55 degrees, or any suitable number of degrees.

In some embodiments, control circuitry 604 is configured to determine atleast one foreground area of a visual field and at least one peripheralarea of a visual field. Control circuitry 604 may determine a foregroundarea within a fourth number of degrees to the right and left of thecenter of gaze of the user, a fifth number of degrees above the centerof gaze of the user, and a sixth number of degrees above the center ofgaze of the user. The fourth, fifth, and sixth numbers may be smallerthan the first, second, and third numbers, respectively. In someembodiments, the fourth, fifth, and sixth numbers of degrees are equalto or greater than 18 degrees, for example, 20 degrees, 25 degrees, 30degrees, 35 degrees, >35 degrees, or any suitable number of degrees. Insome embodiments, the fourth, fifth, and sixth numbers of degrees areless than 18 degrees, for example, 15 degrees, 10 degrees, 5 degrees, <5degrees, or any suitable number of degrees.

In some implementations, control circuitry 604 is configured to accountfor visual impairments of a user in determining the visual field,foreground area, and/or peripheral area of the user. For example, if theuser is blind in one eye, or has a visual impairment in one eye, controlcircuitry 604 may determine the visual field, foreground area, andperipheral area to only extend within a certain number of degrees of theunimpaired eye. As another example, if the peripheral vision of the useris impaired, control circuitry 604 may determine the peripheral area ofthe user to be smaller than for users without impaired peripheralvision. Control circuitry 604 may determine the visual field, foregroundarea, and peripheral area of each user using parameters specific to eachuser. For example, control circuitry 604 may account for a blind spot inan eye of a user in determining visual field, foreground area, andperipheral area.

In some implementations, user input interface 610 is configured toreceive an input from the user and, in response to the input, controlcircuitry 604 is configured to generate for display additional contentin a portion of the virtual reality environment corresponding to theforeground area of the visual field of the user. For example, in FIG. 2,upon receiving the input from the user, control circuitry 604 maygenerate for display additional content 122 in a portion (for example,second portion 110) of virtual reality environment 104 corresponding toforeground area 130 of second visual field 126. The user input may beany type of input, such as input via a remote control, mouse, trackball,keypad, keyboard, touch screen, touchpad, stylus input, joystick, voicerecognition interface, or any other user input interface. For example,the user input may consist of the user saying, “Show me the additionalcontent” or a similar statement. This feature may be beneficial if, forexample, the user desires to be able to see the additional contenteasily in the foreground area of the user's visual field, and is notconcerned about the additional content interfering with the user'sviewing of main content.

In some implementations, user equipment device 600 further consists of atimer. The timer may be configured to measure a time period after amovement of a center of gaze of a user during which the center of gazeof a user has not substantially moved. For example, after a movement ofthe center of gaze of the user, the timer may be configured to measure atime during which the center of gaze of the user has not moved more than1 mm, 5 mm, 1 cm, 10 cm, 100 cm, >100 cm, or any other suitabledistance. Control circuitry 604 may be further configured to compare thetime period to a threshold time period. For example, the threshold timeperiod may be 1 millisecond, 10 milliseconds, 100 milliseconds, 1second, 5 seconds, 10 seconds, >10 seconds, or any suitable time period.In response to determining that the time period measured by the timer isgreater than the threshold time period, control circuitry 604 may beconfigured to generate for display additional content (for example,additional content 122) on a portion (for example, second portion 110)of a virtual reality environment (for example, virtual realityenvironment 104) corresponding to a peripheral area of the user's visualfield (for example, peripheral area 132 of second visual field 126). Inresponse to determining that the time period measured by the timer isless than the threshold time period, control circuitry 604 may beconfigured to not change the position of the additional content.Measuring the time period during which the center of gaze of the userhas not substantially moved after a movement of the center of gaze ofthe user allows the control circuitry 604 to not move the additionalcontent if the center of gaze of the user is moving rapidly. Forexample, if the center of gaze of the user is moving rapidly, it may beconfusing and/or distracting for the user if the additional content alsomoves rapidly in response to the rapid movements of the center of gaze.As another example, the user may wish to look at the additional contentfor a short period of time (for example, 1 millisecond, 10 milliseconds,100 milliseconds, 1 second, 5 seconds, 10 seconds, >10 seconds, etc.)with main vision and then look back at main content on the virtualreality environment. For example, if the additional content shows avideo of the user's physical surroundings, the user may wish to quicklycheck on something in the physical surroundings, such as a nearby child,and then focus once more on main content. In such a situation, it may behelpful for the additional content to remain in place, rather thanmoving when the center of gaze of the user moves to look at theadditional content and then quickly moves back to the main content.

In some embodiments, when the additional content is generated fordisplay in a portion of the virtual reality environment corresponding toa peripheral area of the user's visual field, the additional content isgenerated for display at a lower image and/or video quality (forexample, lower resolution, frame rate, etc.) than main content shown bythe virtual reality environment. This can help to conserve power,memory, bandwidth, etc., of a heads up display (for example, heads updisplay 102) showing the virtual reality environment. Additionally,generating for display the additional content at a lower image and/orvideo quality may not be detrimental to the user's viewing experiencebecause the user may not be able to differentiate between high and lowquality images and/or video viewed in peripheral areas of the user'svisual field. When the center of gaze of the user moves to look at theadditional content for a short period of time, such that the additionalcontent is in a location corresponding to a foreground area of theuser's visual field, the image and/or video quality of the additionalcontent may be increased. The user may be able to differentiate betweenhigh and low quality images and/or video viewed in foreground areas ofthe user's visual field. By generating for display the additionalcontent at a higher image and/or video quality, it is ensured that theuser's viewing of the additional content is not compromised (forexample, by the user needing to view a low quality version of theadditional content). When the center of gaze of the user moves back fromthe additional content to the main content, the image and/or videoquality of the additional content may be decreased again.

In some implementations, user input interface 610 is configured toreceive an input from the user and, in response to the input, controlcircuitry 604 is configured to remove additional content from a heads updisplay. For example, in FIG. 4, in response to the input, controlcircuitry 604 may remove additional content 222 from heads up display222. The user input may be any type of input, such as input via a remotecontrol, mouse, trackball, keypad, keyboard, touch screen, touchpad,stylus input, joystick, voice recognition interface, or any other userinput interface. For example, the user input may consist of the usersaying, “Remove the additional content” or a similar statement. Thisfeature may be beneficial if, for example, the user does not requireassistance from the additional content in performing a full bodymovement and prefers to view content on the heads up display withoutinterference from the additional content. In some implementations, inresponse to the input, control circuitry 604 may move the additionalcontent to a portion of the virtual reality environment corresponding toa peripheral area of the visual field of the user.

In some implementations, user input interface 610 is configured topresent an option to the user to stop playback of main content beingdisplayed on a heads up display. The option may be any type of output,such as output via a display or speaker. For example, the option mayconsist of text on a display reading, “Do you wish to stop playback ofthe video?” or a similar statement. This feature may be beneficial if,for example, the user desires to focus on performing a full bodymovement and does not want to miss content being displayed on the headsup display. Therefore, the feature provides, for example, an option tothe user to pause playback of a media asset or any other type of contentand then resume playback once the user has completed the fully bodymovement. Playback of the main content may resume automatically once itis detected that the user has completed the full body movement, or userinput interface 610 may present another option to the user to resumeplayback of the main content.

FIG. 13 is a flowchart of illustrative steps for presenting additionalcontent in virtual reality environments on heads up displays showingmain content without interfering with a user's viewing of the maincontent.

The main content may be any content that is intended to be the object ofa user's main focus. For example, the main content may be a media asset,such as a movie, a television show, a video game, or a virtual realityworld. As another example, the main content may be a media guidancescreen.

The additional content may be any content that is not main content. Theadditional content may be unrelated to the main content or related tothe main content. For example, the additional content may be a video ofthe user's physical surroundings, stock price information, sports scoreinformation, news information, weather information, a clock, or aschedule of events.

The virtual reality environment may be non-physical content displayed toa user in such a way that the non-physical content appears to the userto have a semblance of physicality. For example, the virtual realityenvironment may be a virtual world (for example, a virtual world in agame) which appears to the user to be the world in which user islocated. As another example, the virtual reality environment may benon-physical content that appears to the user to be superimposed on thephysical world. For example, the virtual reality environment may be aspeedometer display (or any other display) that is superimposed on whatthe user sees through the windshield of his or her car (or any othertransparent surface). As another example, the virtual realityenvironment may be a media asset (for example, a television show or amovie) presented to the user such that the display of the media assetfully encompasses the visual field of the user.

The heads up display may be any display capable of displayingnon-physical content to a user in such a way that the non-physicalcontent appears to the user to have a semblance of physicality. Forexample, the heads up display may be a head-mounted display that fullycovers the eyes of the user. The head-mounted display may be configuredas eyeglasses, binoculars, a helmet, etc. As another example, the headsup display may be a display (for example, a display integrated with awindshield or eyeglasses) that superimposes non-physical content on aview of the physical world which the user can see through the heads updisplay. As another example, the heads up display may be a room in whichthe user is located, where the room is covered in display screens.

It should be noted that process 1100, or any step thereof, could occuron, or be provided by, any of the devices shown in FIGS. 8-12. Forexample, process 1100 may be executed by control circuitry 604 (FIG. 8).

In step 1102, first main content is generated for display in a firstportion of a virtual reality environment in a heads up display. Thefirst portion corresponds to a foreground area of a first visual fieldof the user.

The first main content may be any portion of main content. The firstmain content may be a subset of the main content. For example, the firstmain content may be one or more objects or characters in a media asset.As another example, the first main content may be a particular areawithin a virtual world. As another example, the first main content maybe a specific portion of a media guidance screen.

The first portion of the virtual reality environment may be any portionof the virtual reality environment. The first portion may be a subset ofthe virtual reality environment. The first portion may be a top, bottom,right, or left portion of the virtual reality environment. The firstportion may be approximately a majority or approximately a minority ofthe virtual reality environment.

A visual field of the user may be anything that a user can see when theuser is in a specific position. For example, the visual field may bedetermined based on movements of the user's head. As another example,the visual field may be determined based on movements of the user'scenter of gaze. For example, the visual field of the user may encompassareas within a first number of degrees to the right and left of theuser's center of gaze, a second number of degrees above the center ofgaze, and a third number of degrees below the center of gaze. Forexample, the first number of degrees may equal to or greater than 95degrees, for example, 95 degrees, 100 degrees, 105 degrees, 110 degrees,115 degrees, 120 degrees, >120 degrees, or any suitable number ofdegrees. Alternatively, the first number of degrees may be less than 95degrees, for example, 90 degrees, 85 degrees, 80 degrees, 75 degrees, 70degrees, <70 degrees, or any suitable number of degrees. For example,the second number of degrees may be equal to or greater than 60 degrees,for example, 65 degrees, 70 degrees, 75 degrees, 80 degrees, 85degrees, >85 degrees, or any suitable number of degrees. Alternatively,the second number of degrees may be less than 60 degrees, for example,55 degrees, 50 degrees, 45 degrees, 40 degrees, <40 degrees, or anysuitable number of degrees. For example, the third number of degrees maybe equal to or greater than 70 degrees, for example, 75 degrees, 80degrees, 85 degrees, 90 degrees, >90 degrees, or any suitable number ofdegrees. Alternatively, the third number of degrees may be less than 70degrees, for example, 65 degrees, 60 degrees, 55 degrees, <55 degrees,or any suitable number of degrees. The visual field may be the portionsof a screen of a head-mounted display that the user can see. The visualfield of the user may vary from user to user and may depend on visualimpairments specific to each user.

A foreground area of a user's visual field may be any portion of theuser's visual field that the user can see with normal vision. Theforeground area may encompass a subset of the visual field. Theforeground area may encompass areas of the visual field that aresubstantially in the center of the user's visual field. The foregroundarea may be determined based on movements of the user's head. As anotherexample, the foreground area may be determined based on movements of theuser's center of gaze. For example, the foreground area may be within afourth number of degrees to the right and left of the center of gaze ofthe user, a fifth number of degrees above the center of gaze of theuser, and a sixth number of degrees below the center of gaze of theuser. For example, the fourth, fifth, and sixth numbers of degrees maybe equal to or greater than 18 degrees, for example, 20 degrees, 25degrees, 30 degrees, 35 degrees, >35 degrees, or any suitable number ofdegrees. Alternatively, the fourth, fifth, and sixth numbers of degreesmay be less than 18 degrees, for example, 15 degrees, 10 degrees, 5degrees, <5 degrees, or any suitable number of degrees. The foregroundarea may be the portions of a screen of a head-mounted display that theuser can see with main vision. The foreground area may vary from user touser and may depend on visual impairments specific to each user.

In step 1104, second main content and additional content are generatedfor display in a second portion of the virtual reality environment inthe heads up display. The second portion corresponds to a peripheralarea of the first visual field of the user.

The second portion of the virtual reality environment may be any portionof the virtual reality environment. The second portion may be a subsetof the virtual reality environment. The second portion may be differentthan the first portion. The second portion may be a top, bottom, right,or left portion of the virtual reality environment. The second portionmay be near or at the boundary or corner of the virtual realityenvironment. The second portion may be approximately a majority orapproximately a minority of the virtual reality environment.

The second main content may be any portion of main content. The secondmain content may be a subset of the main content. For example, thesecond main content may be one or more objects or characters in a mediaasset. As another example, the second main content may be a particulararea within a virtual world. As another example, the second main contentmay be a specific portion of a media guidance screen.

A peripheral area of a user's visual field may be any portion of auser's visual field that the user can see with peripheral vision. Theperipheral area may be areas that are substantially at the edges of theuser's visual field. The peripheral area may be determined based onmovements of the user's head. As another example, the peripheral fieldmay be determined based on movements of the user's center of gaze. Forexample, the peripheral area may be any portion of the visual field thatis not foreground area. For example, the peripheral area may encompassareas of the visual field within a certain number of degrees of theouter boundary of the visual field. The peripheral area may be theportions of a screen of a head-mounted display that the user can seewith peripheral vision. The peripheral area may vary from user to userand may depend on visual impairments specific to each user.

In step 1106, a movement of a center of gaze of the user is detected.The center of gaze may be any area of the user's visual field to whichthe gaze of the user is substantially focused. The center of gaze may bea central portion of what a user can see. The center of gaze may be amidpoint between the gaze point of each eye of the user. In users with avisual impairment in one eye, the center of gaze may be the gaze pointof the unimpaired eye. The movement of the center of gaze may bedetected by detecting a gaze point of each eye of the user using light.The movement of the center of gaze may be detected by detecting amovement of the head of the user. The movement of the center of gaze maybe detected by detecting a change in a video of the physicalsurroundings of the user with captured with a camera physically coupledto the user.

In step 1108, a second visual field of the user is determined based onthe movement of the center of gaze of the user. For example, the controlcircuitry may determine the second visual field by determining the newcenter of gaze. For example, the control circuitry may determine thesecond visual field by detecting a movement of the head of the user.

In step 1110, it is determined that the second portion corresponds to aforeground area of the second visual field. For example, the secondportion may correspond to a corner of the first visual field butcorrespond to the center of the second visual field.

In step 1112, in response to determining that the second portioncorresponds to the foreground area of the second visual field, theadditional content is generated for display in a third portion of thevirtual reality environment, wherein the third portion corresponds to aperipheral area of the second visual field. The third portion of thevirtual reality environment may be any portion of the virtual realityenvironment. The third portion may be a subset of the virtual realityenvironment. The third portion may be different than the first andsecond portions. The third portion may be a top, bottom, right, or leftportion of the virtual reality environment. The third portion may benear or at the boundary or corner of the virtual reality environment.The third portion may be approximately a majority or approximately aminority of the virtual reality environment.

In some embodiments, detecting the movement of the center of gazeincludes detecting the center of gaze of the user. In certainembodiments, detecting the center of gaze of the user includestransmitting light to each eye of the user; collecting an image of eacheye of the user; detecting, in each image, a location of a reflection inan eye of the user; determining a location of each pupil of the user;comparing the location of each pupil to the location of each reflection;based on comparing the location of each pupil to the location of eachreflection, determining a gaze point of each eye of the user; anddetermining the center of gaze by determining a midpoint between thegaze point of each eye of the user.

In certain embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the head-mounted displayincludes an accelerometer, and detecting the movement of the center ofgaze of the user includes detecting, by the accelerometer, anacceleration of the head of the user.

In some embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the head-mounted displayincludes a camera feeding a video of the user's physical surroundings tothe display, and generating for display the additional content comprisesgenerating for display the video of the user's physical surroundings.

In certain embodiments, generating for display the additional contentincludes generating for display the additional content as apicture-in-picture.

In some embodiments, determining the second visual field of the userbased on the movement of the center of gaze includes determining a newcenter of gaze based on the movement of the center of gaze anddetermining an area that is within a first number of degrees to theright and left of the new center of gaze, a second number of degreesabove the new center of gaze, and a third number of degrees below thenew center of gaze.

In certain embodiments, determining that the second portion correspondsto the foreground area of the second visual field comprises determiningthe foreground area of the second visual field, and determining theforeground area of the second visual field comprises determining an areathat is within a fourth number of degrees to the right and left of thenew center of gaze, a fifth number of degrees above the new center ofgaze, and a sixth number of degrees below the new center of gaze,wherein the fourth number is smaller than the first number, the fifthnumber is smaller than the second number, and the sixth number issmaller than the third number.

In some embodiments, the process further includes receiving an inputfrom the user and, in response to the input, generating for display theadditional content in a portion of the virtual reality environmentcorresponding to a foreground area of the second visual field.

In certain embodiments, generating for display the additional contentincludes measuring a time period after the movement of the center ofgaze of the user during which the center of the gaze of the user has notsubstantially moved; determining that the time period is greater than athreshold time period; and in response to determining that the timeperiod is greater than the threshold time period, generating for displaythe additional content.

FIG. 14 is a flowchart of illustrative steps for enabling users toperform full body movements while viewing virtual reality environmentson heads up displays without interfering with viewing of content on theheads up displays. The virtual reality environment may be non-physicalcontent displayed to a user in such a way that the non-physical contentappears to the user to have a semblance of physicality. For example, thevirtual reality environment may be a virtual world (for example, avirtual world in a game) which appears to the user to be the world inwhich user is located. As another example, the virtual realityenvironment may be non-physical content that appears to the user to besuperimposed on the physical world. For example, the virtual realityenvironment may be a speedometer display (or any other display) that issuperimposed on what the user sees through the windshield of his or hercar (or any other transparent surface). As another example, the virtualreality environment may be a media asset (for example, a television showor a movie) presented to the user such that the display of the mediaasset fully encompasses the visual field of the user.

The heads up display may be any display capable of displayingnon-physical content to a user in such a way that the non-physicalcontent appears to the user to have a semblance of physicality. Forexample, the heads up display may be a head-mounted display that fullycovers the eyes of the user. The head-mounted display may be configuredas eyeglasses, binoculars, a helmet, etc. As another example, the headsup display may be a display (for example, a display integrated with awindshield or eyeglasses) that superimposes non-physical content on aview of the physical world which the user can see through the heads updisplay. As another example, the heads up display may be a room in whichthe user is located, where the room is covered in display screens.

The full body movement may be any physical movement by a user thatrequires movement of a substantial portion of the user's entire body.For example, the full body movement may be walking, jumping, standingup, sitting down, rotating one's body, etc.

It should be noted that process 1200, or any step thereof, could occuron, or be provided by, any of the devices shown in FIGS. 8-12. Forexample, process 1200 may be executed by control circuitry 604 (FIG. 8).

In step 1202, main content is generated for display in a first portionof a virtual reality environment in a heads up display. The firstportion corresponds to a foreground area of a visual field of the user.

The first portion of the virtual reality environment may be any portionof the virtual reality environment. The first portion may be a subset ofthe virtual reality environment. The first portion may be a top, bottom,right, or left portion of the virtual reality environment. The firstportion may be approximately a majority or approximately a minority ofthe virtual reality environment.

The main content may be content that is intended to be the object of auser's main focus. For example, the main content may be a media asset,such as a movie, a television show, a video game, or a virtual realityworld. As another example, the main content may be a media guidancescreen.

A visual field of the user may be anything that a user can see when theuser is in a specific position. For example, the visual field may bedetermined based on movements of the user's head. As another example,the visual field may be determined based on movements of the user'scenter of gaze. For example, the visual field of the user may encompassareas within a first number of degrees to the right and left of theuser's center of gaze, a second number of degrees above the center ofgaze, and a third number of degrees below the center of gaze. Forexample, the first number of degrees may equal to or greater than 95degrees, for example, 95 degrees, 100 degrees, 105 degrees, 110 degrees,115 degrees, 120 degrees, >120 degrees, or any suitable number ofdegrees. Alternatively, the first number of degrees may be less than 95degrees, for example, 90 degrees, 85 degrees, 80 degrees, 75 degrees, 70degrees, <70 degrees, or any suitable number of degrees. For example,the second number of degrees may be equal to or greater than 60 degrees,for example, 65 degrees, 70 degrees, 75 degrees, 80 degrees, 85degrees, >85 degrees, or any suitable number of degrees. Alternatively,the second number of degrees may be less than 60 degrees, for example,55 degrees, 50 degrees, 45 degrees, 40 degrees, <40 degrees, or anysuitable number of degrees. For example, the third number of degrees maybe equal to or greater than 70 degrees, for example, 75 degrees, 80degrees, 85 degrees, 90 degrees, >90 degrees, or any suitable number ofdegrees. Alternatively, the third number of degrees may be less than 70degrees, for example, 65 degrees, 60 degrees, 55 degrees, <55 degrees,or any suitable number of degrees. The visual field may be the portionsof a screen of a head-mounted display that the user can see. The visualfield may vary from user to user and may depend on visual impairmentsspecific to each user.

A foreground area of a user's visual field may be any portion of theuser's visual field that the user can see with normal vision. Theforeground area may encompass a subset of the visual field. Theforeground area may encompass areas of the visual field that aresubstantially in the center of the user's visual field. The foregroundarea may be determined based on movements of the user's head. As anotherexample, the foreground area may be determined based on movements of theuser's center of gaze. For example, the foreground area may be within afourth number of degrees to the right and left of the center of gaze ofthe user, a fifth number of degrees above the center of gaze of theuser, and a sixth number of degrees below the center of gaze of theuser. For example, the fourth, fifth, and sixth numbers of degrees maybe equal to or greater than 18 degrees, for example, 20 degrees, 25degrees, 30 degrees, 35 degrees, >35 degrees, or any suitable number ofdegrees. Alternatively, the fourth, fifth, and sixth numbers of degreesmay be less than 18 degrees, for example, 15 degrees, 10 degrees, 5degrees, <5 degrees, or any suitable number of degrees. The foregroundarea may be the portions of a screen of a head-mounted display that theuser can see with main vision. The foreground area may vary from user touser and may depend on visual impairments specific to each user.

In step 1204, additional content is generated for display in a secondportion of the virtual reality environment in the heads up display. Thesecond portion corresponds to a peripheral area of the visual field ofthe user.

The second portion of the virtual reality environment may be any portionof the virtual reality environment. The second portion may be a subsetof the virtual reality environment. The second portion may be differentthan the first portion. The second portion may be a top, bottom, right,or left portion of the virtual reality environment. The second portionmay be near or at the boundary or corner of the virtual realityenvironment. The second portion may be approximately a majority orapproximately a minority of the virtual reality environment.

The additional content assists the user to perform the full bodymovement. For example, the additional content may be a video of theuser's physical surroundings. As another example, the additional contentmay be a map of the user's physical surroundings.

A peripheral area of a user's visual field may be any portion of auser's visual field that the user can see with peripheral vision. Theperipheral area may be areas that are substantially at the edges of theuser's visual field. The peripheral area may be determined based onmovements of the user's head. As another example, the peripheral fieldmay be determined based on movements of the user's center of gaze. Forexample, the peripheral area may be any portion of the visual field thatis not foreground area. For example, the peripheral area may encompassareas of the visual field within a certain number of degrees of theouter boundary of the visual field. The peripheral area may be theportions of a screen of a head-mounted display that the user can seewith peripheral vision. The peripheral area may vary from user to userand may depend on visual impairments specific to each user.

In step 1206, a full body movement of the user is detected. For example,the full body movement may be detected by detecting an acceleration of aportion of the body of the user. For example, the detection module maybe detected by detecting a footstep. As another example, the full bodymovement may be detected by detecting a change in a video of thephysical surroundings of the user captured with a camera physicallycoupled to the user.

In step 1208, in response to detecting the full body movement of theuser, the additional content is generated for display in the firstportion of the virtual reality environment. The additional contentassists the user to perform the full body movement. For example, theadditional content may be a video of the user's physical surroundings.As another example, the additional content may be a map of the user'sphysical surroundings.

In some embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the head-mounted displayincludes a camera feeding a video of the user's physical surroundings tothe display, and generating for display the additional content comprisesgenerating for display the video of the user's physical surroundings.

In certain embodiments, generating for display the additional contentcomprises generating for display the additional content as apicture-in-picture.

In some embodiments, the heads up display is a head-mounted displayphysically coupled to the head of the user, the head-mounted displayincludes an accelerometer, and detecting the full body movement of theuser comprises detecting, by the accelerometer, a first footstep takenby the user.

In certain embodiments, the process 1300 further includes detecting asecond footstep taken by the user and, in response to the detecting ofthe second footstep, enlarging the additional content.

In some embodiments, the process 1300 further includes detecting asecond footstep taken by the user and, in response to the detecting ofthe second footstep, performing at least one of decreasing an opacity ofthe main content and increasing an opacity of the additional content.

In certain embodiments, the process 1300 further includes receiving aninput from the user and, in response to the input, removing theadditional content from the heads up display.

In some embodiments, detecting the full body movement of the usercomprises detecting a change in the video of the user's physicalsurroundings.

In certain embodiments, the process 1300 further includes detecting thatthe user is substantially stationary and, in response to the detectingthat the user is substantially stationary, generating for displaying theadditional content in a third portion of the display.

In some embodiments, the process 1300 further includes presenting anoption to the user to stop playback of the main content.

FIGS. 15 and 16 present processes for control circuitry (e.g., controlcircuitry 604) to present additional content in virtual realityenvironments on heads up displays showing main content withoutinterfering with a user's viewing of the main content in accordance withsome embodiments of the disclosure. In some embodiments this algorithmmay be encoded on to a non-transitory storage medium (e.g., storagedevice 608) as a set of instructions to be decoded and executed byprocessing circuitry (e.g., processing circuitry 606). Processingcircuitry may in turn provide instructions to other sub-circuitscontained within control circuitry 604, such as the tuning, videogenerating, encoding, decoding, encrypting, decrypting, scaling,analog/digital conversion circuitry, and the like.

The flowchart in FIG. 15 describes a process 1300 implemented on controlcircuitry (e.g., control circuitry 604) to present additional content invirtual reality environments on heads up displays showing main contentwithout interfering with a user's viewing of the main content inaccordance with some embodiments of the disclosure.

At step 1302, control circuitry 604 will begin to detect a movement of acenter of gaze of a user. In some embodiments, this may be done eitherdirectly or indirectly in response to a user action or input (e.g., fromsignals received by control circuitry 604, user input interface 610, ordetection module 616). For example, the process may begin directly inresponse to control circuitry 604 receiving signals from detectionmodule 616.

At step 1304, control circuitry 604 proceeds to determine valuesdescribing a location on the heads up display corresponding to aforeground area of a visual field of the user based on the movement ofthe center of gaze.

At step 1306, control circuitry 604 proceeds to retrieve the currentinstance of values describing the location of the additional content onthe heads up display. In some embodiments control circuitry 604 mayreceive a single primitive data structure that represents the locationof the additional content on the heads up display. In some embodimentsthe values describing the location of the additional content on theheads up display may be stored as part of a larger data structure, andcontrol circuitry 604 may retrieve the values by executing appropriateaccessor methods to retrieve the values from the larger data structure.

At step 1308, control circuitry 604 proceeds to compare the valuesdescribing the location on the heads up display corresponding to theforeground area of the visual field of the user to the values describingthe location of the additional content on the heads up display. In someembodiments, control circuitry 604 may directly compare the valuesdescribing the location on the heads up display corresponding to theforeground area of the visual field of the user to the values describingthe location of the additional content on the heads up display byaccessing the values respectively and performing a value comparison. Insome instances, control circuitry 604 may call a comparison function(e.g., for object to object comparison) to compare the values describingthe location on the heads up display corresponding to the foregroundarea of the visual field of the user to the values describing thelocation of the additional content on the heads up display.

At step 1310, control circuitry 604 compares the values describing thelocation on the heads up display corresponding to the foreground area ofthe visual field of the user to the values describing the location ofthe additional content on the heads up display to determine if thelocation of the additional content on the heads up display is within thelocation on the heads up display corresponding to the foreground area ofthe visual field of the user. If the condition is not satisfied, thealgorithm may proceed back to step 1302; if the condition is satisfied,the algorithm may proceed to step 1312 instead.

At step 1312, control circuitry 604 executes a subroutine to generatefor display the additional content in a location on the heads up displaythat corresponds to a peripheral area of the visual field of the user.After the subroutine is executed, the algorithm may proceed back to step1302.

It is contemplated that the descriptions of FIG. 15 may be used with anyother embodiment of this disclosure. In addition, the descriptionsdescribed in relation to the algorithm of FIG. 15 may be done inalternative orders or in parallel to further the purposes of thisdisclosure. For example, conditional statements and logical evaluations,such as those at 1310, may be performed in any order or in parallel orsimultaneously to reduce lag or increase the speed of the system ormethod. Furthermore, it should be noted that the process of FIG. 15 maybe implemented on a combination of appropriately configured software andhardware, and that any of the devices or equipment discussed in relationto FIGS. 8-12 could be used to implement one or more portions of theprocess.

The pseudocode in FIG. 16 describes a process 1400 to present additionalcontent in virtual reality environments on heads up displays showingmain content without interfering with a user's viewing of the maincontent in accordance with some embodiments of the disclosure. It willbe evident to one skilled in the art that the process described by thepseudocode in FIG. 16 may be implemented in any number of programminglanguages and a variety of different hardware, and that the style andformat should not be construed as limiting, but rather a generaltemplate of the steps and procedures that would be consistent with codeused to implement some embodiments of this disclosure.

At line 1401, control circuitry 604 runs a subroutine to initializevariables and prepare to present additional content in virtual realityenvironments on heads up displays showing main content withoutinterfering with a user's viewing of the main content. For example, insome embodiments control circuitry 604 may copy instructions fromnon-transitory storage medium (e.g., storage device 608) into RAM orinto the cache for processing circuitry 606 during the initializationstage. Additionally, in some embodiments the values describing thelocation on the heads up display corresponding to a foreground area of avisual field of the user being used for comparison, or a tolerance levelfor determining if two values are essentially equivalent, may beretrieved, set, and stored at 1401.

At line 1405, control circuitry 604 receives instances of the locationon the heads up display corresponding to the foreground area of thevisual field of the user. Control circuitry 604 may receive instances ofthe location on the heads up display corresponding to the foregroundarea of the visual field of the user by receiving, for example, apointer to an array of values describing the location on the heads updisplay corresponding to the foreground area of the visual field of theuser. In another example, control circuitry 604 may receive an object ofa class, such as an iterator object containing elements of the valuesdescribing the location on the heads up display corresponding to theforeground area of the visual field of the user.

At line 1407, control circuitry 604 stores the values describing thelocation on the heads up display corresponding to the foreground area ofthe visual field of the user into a temporary variable “A.” In someembodiments the values describing the location on the heads up displaycorresponding to the foreground area of the visual field of the userwill be stored as part of a larger data structure or class, and thevalues describing the location on the heads up display corresponding tothe foreground area of the visual field of the user may be obtainedthrough appropriate accessor methods. In some embodiments the valuesdescribing the location on the heads up display corresponding to heforeground area of the visual field of the user may be converted from astring or other non-numeric data type into a numeric data type by meansof an appropriate hashing algorithm. In some embodiments, controlcircuitry 604 may call a function to perform a comparison of the valuesdescribing the location on the heads up display corresponding to heforeground area of the visual field of the user to values describing thelocation of the additional content on the heads up display. In someembodiments the values describing the location on the heads up displaycorresponding to the foreground area of the visual field of the user maybe encoded as a primitive data structure, and rather than using atemporary variable, the values describing the location on the heads updisplay corresponding to the foreground area of the visual field of theuser may be directly used in the comparison at line 1409.

At line 1408, control circuitry 604 stores the values describing thelocation of the additional content on the heads up display into atemporary variable “B.” Similar to the values describing the location onthe heads up display corresponding to the foreground area of the visualfield of the user, in some embodiments the values describing thelocation of the additional content on the heads up display will bestored as part of a larger data structure or class, and the valuesdescribing the location of the additional content on the heads updisplay may be obtained through accessor methods. In some embodimentsthe values describing the location of the additional content on theheads up display may be converted from a string or other non-numericdata type into a numeric data type by means of an appropriate hashingalgorithm, or the values describing the location of the additionalcontent on the heads up display may be a primitive data structure, andmay be directly used in the comparison at line 1409.

At line 1409, control circuitry 604 compares the values describing thelocation on the heads up display corresponding to the foreground area ofthe visual field of the user to the values describing the location ofthe additional content on the heads up display to determine if thelocation of the additional content on the heads up display is within thelocation on the heads up display corresponding to a foreground area ofthe visual field of the user. If values describing a location on theheads up display describe the top, bottom, right and left boundaries ofthe location on the heads up display, and if values increase asboundaries proceed from left to right and bottom to top, determiningwhether the location of the additional content on the heads up displayis within the location on the heads up display corresponding to theforeground area of the visual field of the user is achieved as follows.The value describing the right boundary of the location of theadditional content on the heads up display is subtracted from the valuedescribing the right boundary of the location on the heads up displaycorresponding to the foreground area of the visual field of the user.The value describing the top boundary of the location of the additionalcontent on the heads up display is subtracted from the value describingthe top boundary of the location on the heads up display correspondingto the foreground area of the visual field of the user. The valuedescribing the left boundary of the location on the heads up displaycorresponding to the foreground area of the visual field of the user issubtracted from the value describing the left boundary of the locationof the additional content on the heads up display. The value describingthe bottom boundary of the location on the heads up displaycorresponding to the foreground area of the visual field of the user issubtracted from the value describing the bottom boundary of the locationof the additional content on the heads up display. The value of eachdifference is calculated, and value of each difference is compared to apredetermined tolerance level. In some embodiments the tolerance levelmay be a set percentage of either the values describing the location onthe heads up display corresponding to the foreground area of the visualfield of the user or the values describing the location of theadditional content on the heads up display. In some embodiments thetolerance level may be a fixed number. For example, setting thetolerance level to a set multiple of machine epsilon may allow for thealgorithm to account for small rounding errors that may result from theuse of floating point arithmetic. In some embodiments the tolerancelevel may be set to zero.

At line 1410, control circuitry 604 executes a subroutine to change thelocation of the additional content on the heads up display if any of thevalues of each difference are greater than the predetermined tolerancelevel. In some embodiments this may be achieved by processing circuitry606 sending the appropriate signals to display 612.

At line 1412, control circuitry 604 runs a termination subroutine afterthe algorithm has performed its function. For example, in someembodiments control circuitry 604 may destruct variables, performgarbage collection, free memory or clear the cache of processingcircuitry 606.

It will be evident to one skilled in the art that process 1400 describedby the pseudocode in FIG. 16 may be implemented in any number ofprogramming languages and a variety of different hardware, and theparticular choice and location of primitive functions, logicalevaluations, and function evaluations are not intended to be limiting.It will also be evident that the code may be refactored or rewritten tomanipulate the order of the various logical evaluations, perform severaliterations in parallel rather than in a single iterative loop, or tootherwise manipulate and optimize run-time and performance metricswithout fundamentally changing the inputs or final outputs. For example,in some embodiments break conditions may be placed after certain linesto speed operation, or the conditional statements may be replaced with acase-switch.

The above-described embodiments of the present disclosure are presentedfor purposes of illustration and not of limitation, and the presentdisclosure is limited only by the claims that follow. Furthermore, itshould be noted that the features and limitations described in any oneembodiment may be applied to any other embodiment herein, and flowchartsor examples relating to one embodiment may be combined with any otherembodiment in a suitable manner, done in different orders, or done inparallel. In addition, the systems and methods described herein may beperformed in real time. It should also be noted that the systems and/ormethods described above may be applied to, or used in accordance with,other systems and/or methods.

I claim:
 1. A method for viewing content on a heads up display,comprising: generating for display main content in a first portion ofthe heads up display, wherein the first portion corresponds to anon-peripheral area of a visual field of a user; generating for display,in a second portion of the heads up display, additional content, whereinthe second portion corresponds to a peripheral area of the visual fieldof the user; detecting, with control circuitry, a head movement of theuser based on a signal from an accelerometer; and in response todetecting the head movement with the control circuitry, moving displayof the additional content from the second portion of the heads updisplay to the first portion of the heads up display.
 2. The method ofclaim 1, wherein generating for display the additional content comprisesgenerating for display the additional content as a picture-in-picture.3. The method of claim 1, wherein detecting the head movement includesdetecting a full body movement of the user.
 4. The method of claim 3,wherein detecting the full body movement of the user comprisesdetecting, by the accelerometer, a first footstep taken by the user. 5.The method of claim 4, further comprising: detecting a second footsteptaken by the user; and in response to the detecting of the secondfootstep, enlarging the additional content.
 6. The method of claim 4,further comprising: detecting a second footstep taken by the user; andin response to the detecting of the second footstep, performing at leastone of decreasing an opacity of the main content and increasing anopacity of the additional content.
 7. The method of claim 1, furthercomprising: receiving an input from the user; and in response to theinput, removing the additional content from the heads up display.
 8. Themethod of claim 1, wherein the heads up display is a head-mounteddisplay physically coupled to the head of the user.
 9. The method ofclaim 1, further comprising: detecting that the user is substantiallystationary; and in response to the detecting that the user issubstantially stationary, generating for display the additional contentin a third portion of the display that is not in the non-peripheral areaof the visual field of the user.
 10. The method of claim 1, furthercomprising presenting an option to the user to stop playback of the maincontent.
 11. A system for viewing content on a heads up display,comprising: control circuitry configured to: generate for display maincontent in a first portion of a heads up display, wherein the firstportion corresponds to a non-peripheral area of a visual field of auser; generate for display, in a second portion of the heads up display,additional content, wherein the second portion corresponds to aperipheral area of the visual field of the user; and in response todetecting a head movement, move display of the additional content fromthe second portion to the first portion; and a detection moduleconfigured to detect the head movement of the user.
 12. The system ofclaim 11, wherein the control circuitry is further configured, whengenerating for display the additional content, to generate for displaythe additional content as a picture-in-picture.
 13. The system of claim11, wherein the detection module is further configured, when detectingthe head movement, to detect a full body movement of the user.
 14. Thesystem of claim 11, wherein the detection module is further configured,when detecting the full body movement of the user, to detect, using theaccelerometer, a first footstep taken by the user.
 15. The system ofclaim 14, wherein the detection module is further configured to detect,using the accelerometer, a second footstep taken by the user, and thecontrol circuitry is further configured, in response to the detecting ofthe second footstep, to enlarge the additional content.
 16. The systemof claim 14, wherein the detection module is further configured todetect, using the accelerometer, a second footstep taken by the user,and the control circuitry is further configured, in response to thedetecting of the second footstep, to perform at least one of decreasingan opacity of the main content and increasing an opacity of theadditional content.
 17. The system of claim 11, wherein a user interfaceis configured to receive an input from the user, and the controlcircuitry is further configured, in response to the input, to remove theadditional content from the heads up display.
 18. The system of claim11, wherein the heads up display is a head-mounted display physicallycoupled to the head of the user.
 19. The system of claim 11, wherein thedetection module is further configured to detect that the user issubstantially stationary, and the control circuitry is furtherconfigured, in response to the detecting that the user is substantiallystationary, to generate for display the additional content in a thirdportion of the display that is not in the non-peripheral area of thevisual field of the user.
 20. The system of claim 11, wherein a userinterface is configured to present an option to the user to stopplayback of the main content.